Traffic Accident Severity Prediction via Large Language Model-Driven Semantic Feature Enhancement

Abstract

Predicting the severity of traffic accidents remains challenging due to the limited ability of existing methods to extract deep semantic information from unstructured accident narratives, as traditional approaches typically depend on structured data alone. This study proposes a severity prediction approach enhanced by semantic risk reasoning derived from large language models (LLMs). A prompt-engineering template is designed to guide LLMs in extracting proxy semantic features from accident descriptions, forming an enriched feature set that incorporates causal logic. These semantic features are fused with traditional structured features through three integration strategies—direct feature concatenation, optimized feature selection, and model-level fusion. Experiments based on 4013 accident records from expressways in Yunnan Province, China, demonstrate that models using LLM-derived semantic features significantly outperform those relying solely on structured features. Notably, the LightGBM model utilizing semantic features within a balanced learning framework achieves a severe accident recall of 77.8%. While model-level fusion proves optimal for XGBoost (improving Macro-F1 to 0.6356), we identify a “feature dilution” effect in other classifiers, where high-quality semantic reasoning is compromised by low-quality structured noise. These findings indicate that the proposed approach effectively enhances the identification of high-risk accidents and offers a novel semantic-aware solution for traffic safety management. Furthermore, the obtained results provide actionable insights for traffic management agencies to optimize emergency response resource allocation and formulate targeted accident prevention strategies.

1. Introduction

Traffic accidents constitute a major global public safety challenge, causing severe threats to human life and producing substantial socioeconomic losses each year. Globally, road traffic accidents have emerged as a critical issue affecting public health and urban governance [1]. Accurately predicting accident severity plays a vital role in improving emergency response efficiency, optimizing rescue resource allocation, identifying high-risk road segments, and enabling targeted prevention strategies [2].

Research on accident severity prediction has evolved from traditional statistical models to machine learning approaches. Early studies predominantly employed logistic regression [3], Bayesian networks [4], and ordered probability models [5], which provide valuable interpretability [6] but require strong assumptions regarding variable distribution and struggle to represent complex nonlinear relationships [7]. Furthermore, visual analysis tools such as road incident maps have also proven to be effective for analyzing traffic safety and highlighting critical high-risk locations [8]. Building upon these analytical foundations, recent review studies have systematically summarized this methodological evolution, highlighting a gradual transition from statistically driven models toward data-driven machine learning frameworks that integrate heterogeneous factors such as vehicle attributes, roadway geometry, driver behavior, and environmental conditions. These reviews emphasize that the increasing complexity of influencing factors has been a key driver for adopting machine learning techniques in accident severity prediction [9]. The development of artificial intelligence has facilitated the integration of machine learning models into this field. Bayesian-optimized random forests improved predictive accuracy and explained feature importance [10], support vector machines effectively addressed high-dimensional classification [11], and hybrid frameworks combining random forests and convolutional neural networks enhanced discrimination capability [12]. More recently, multi-view graph convolution methods reduced prediction error by exploiting structural correlations within heterogeneous data [13]. Deep learning models further demonstrated considerable prediction potential, including multi-layer perceptrons [14], convolution-based fusion architectures [15], and image-based representations for mountainous expressways [16]. Ensemble learning has also provided avenues for performance enhancement [17].

Despite notable progress, existing models largely rely on structured data and insufficiently exploit semantic and causal information embedded in natural language accident records. Initial studies have attempted textual feature extraction to complement structured attributes. Word-embedding-based semantic enhancement [18] and Term Frequency–Inverse Document Frequency (TF-IDF) with random forest classification [19] have verified that textual information contributes to predictive accuracy. However, these approaches primarily capture shallow linguistic representations and struggle to uncover complex causal logic, contextual associations, and critical risk factors embedded in accident narratives [20,21,22,23,24,25]. As a result, distinguishing high-risk accidents remains challenging. In addition, severe accidents typically account for less than 10% of samples in general traffic datasets [21]. However, in the specific dataset used in this study, the imbalance is even more extreme, with moderate and severe accidents combined accounting for less than 3% of the total cases, presenting a significantly harder challenge for classification. This pronounced class imbalance leads to reduced recall for severe cases. Insufficient model interpretability further restricts adoption in operational traffic safety decision-making [22,23].

Recent breakthroughs in large language models (LLMs) [26] provide new opportunities for semantic feature extraction and causal reasoning. Chain-of-thought mechanisms have enhanced transparency in prediction tasks [27], while emerging frameworks such as CrashLLM attempt to transform structured accident data into language sequences to support causal inference [28]. Growing evidence has also highlighted the necessity of reliability and decision-level interpretability when deploying LLM-based systems in safety-critical applications. Customized LLM frameworks have been proposed to enhance trustworthiness and actionable insight in crash pattern prediction and intervention planning [29]. These advances underscore the potential of LLMs to capture richer semantic representations—particularly those related to causal logic and contextual interactions—that conventional methods struggle to model.

Motivated by these observations, this study advances a new perspective that moves beyond traditional “textual representation” to “semantic risk reasoning.” Unlike conventional Natural Language Processing (NLP) methods (e.g., TF-IDF [30] or BERT [31]) that treat accident narratives as static word embeddings—thereby missing the latent causal chains—we conceptualize the Large Language Model not merely as a language processor, but as a Semantic Risk Reasoner. Specifically, we develop a prompt-engineering template to guide LLMs in extracting high-level risk proxies (e.g., injury probability, kinetic impact) from unstructured descriptions. These semantic features are then fused with original structured attributes. To address the “black-box” nature of fusion, we systematically evaluate three integration strategies to reveal how semantic logic interacts with structured data.

Consequently, the contributions of this study are threefold. First, we establish a “Semantic Risk Reasoning” framework that transforms unstructured accident narratives into interpretable, high-dimensional risk probabilities, effectively overcoming the limitations of shallow linguistic embeddings. Second, through multi-model validation, we identify a critical “Feature Dilution” effect, revealing the mechanism by which high-quality semantic reasoning can be compromised by low-quality structured noise in certain classifiers such as LightGBM. Finally, we demonstrate that the proposed approach achieves state-of-the-art recall for severe accidents by synergizing semantic risk reasoning with robust class-balancing strategies, thereby offering a robust solution to the persistent challenge of long-tail distributions in traffic safety.

The remainder of this paper is organized as follows: Section 2 details the proposed LLM-enhanced framework, including the data preprocessing pipeline, the semantic feature generation process using prompt engineering, and the three feature fusion strategies. Section 3 presents the experimental results, specifically the ablation study on XGBoost and the robustness analysis across multiple machine learning classifiers. Section 4 discusses the mechanism of semantic risk reasoning, the observed feature dilution effect, and the model’s performance in addressing class imbalance. Finally, Section 5 concludes the study and outlines future research directions.

2. Materials and Methods

2.1. Overview of the LLM-Enhanced Prediction Framework

To address the limitations of conventional accident severity prediction, particularly the insufficient utilization of semantic information and the low recall rate for severe cases, we propose a prediction framework enhanced by semantic features derived from large language models (LLMs). The framework classifies accident severity into three levels (“minor”, “moderate”, and “severe”) by jointly leveraging structured attributes and unstructured narrative descriptions. As illustrated in Figure 1, the workflow consists of four primary stages.

Figure 1. The overall framework of the proposed accident severity prediction method enhanced by LLM-driven semantic features.

Data preprocessing and feature construction were first conducted to prepare both structured and unstructured information for machine learning. Raw structured variables, including time, weather conditions, road classification, and traffic control status, were cleaned, encoded, and engineered, while narrative accident reports recorded by police officers were collected as the textual source for semantic extraction.

Semantic features were then generated using a large language model guided by a multi-layer prompt engineering template. The template defined system roles, domain constraints, output structure, and scoring criteria, enabling the model to perform contextual reasoning over accident descriptions and produce interpretable proxy features such as risk scores and key contributing factors. The outputs were returned in a standardized JSON schema and parsed into machine-learning-compatible variables (B_Features).

To construct an enhanced feature space, these semantic representations were integrated with the original structured attributes (A_Features). Three fusion strategies—direct concatenation, importance-based feature selection, and model-level fusion—were implemented to evaluate the complementary value of semantic features. Global feature importance was assessed to determine whether LLM-generated semantics compensated for missing elements or implicit context in structured data.

The resulting feature sets were applied to machine learning classifiers, including XGBoost [32], LightGBM [33], Random Forest [34], and CatBoost [35], which independently produced three-level severity predictions. Macro-F1 and recall for severe accidents were adopted to address class imbalance and evaluate performance robustness. This experimental phase assesses both predictive improvement and the ability of semantic features to enhance the identification of high-risk accidents and generalization across models.

Overall, the framework integrates LLM-based semantic reasoning with the efficiency of traditional machine learning classifiers, providing a viable solution to address long-standing limitations in accident severity prediction.

2.2. Data Source and Accident Severity Definition

This study utilizes traffic accident records collected from the event reporting system of Yunnan Transportation Investment and Construction Group, China, covering the period from 1 January 2024, to 18 August 2024. The dataset consists of 4013 accident records, each containing multidimensional information, including temporal, spatial, road environment, operational conditions, vehicle involvement, and narrative report descriptions. The target variable for prediction is accident severity. The severity classification in this study references the standards issued by the Ministry of Public Security of the People’s Republic of China, titled Notice on Revising the Classification Criteria for Road Traffic Accidents [36], with minor adaptations to suit the characteristics of the dataset and classification requirements of this study. The final classification categories are defined as follows: Minor accident: fewer than two minor injuries, and financial loss below CNY 1000; Moderate accident: one to two serious injuries or three or more minor injuries, and financial loss below CNY 30,000; Severe accident: at least one fatality, or financial loss exceeding CNY 30,000. The dataset includes both structured attributes and unstructured textual descriptions. Structured fields cover time, road type, alignment, weather conditions, lateral safety facilities, traffic state, and vehicle information, while unstructured content primarily consists of police dispatch records (“alertdescribe”) and management reports (“ownerreport”), which provide detailed contextual descriptions of incident progression and on-site response. A simplified representation of the dataset schema is shown in Table 1.

Table 1. Description and examples of key fields in the traffic accident dataset.

Field Name	Description	Example
alerttime	Time of police alert	1 January 2024 02:38:00
longitude, latitude	Geographic coordinates	104.184759, 28.433447
accidentcause	Cause of accident	Failure to maintain safe distance
vehicletypename	Vehicle type involved	Passenger car
damagedvehnum	Number of damaged vehicles	1
weather	Weather condition	1 = Sunny; 2 = Cloudy; 3 = Rainy
trafficstate	Traffic operation state	1 = Normal traffic flow; 2 = Partially restricted or slow-moving traffic; 3 = Full closure or traffic interruption;
lanename	Lane type	Emergency lane
roadalignmentname	Road alignment	Straight section
locationtypename	Road location type	Tunnel
roadsurfacename	Road surface condition	Dry
roadsidesafetydevicename	Roadside safety facilities	Crash barrier/Retaining wall
alertdescribe	Police dispatch report description	“At 02:38 on 1 January 2024, a passenger vehicle experienced self-ignition on Shuima Expressway, causing temporary traffic suspension in the direction of Zhaotong due to emergency handling requirements. Vehicles were retained near the tunnel entrance for control.”

2.3. Data Preprocessing and Feature Engineering

To construct a unified feature space integrating structured attributes and LLM-generated semantic features, we developed a comprehensive preprocessing pipeline. This workflow comprises data cleaning, feature construction, encoding and normalization, feature selection, and dataset partitioning.

2.3.1. Data Cleaning and Preprocessing

To ensure data reliability and consistency, missing values in numerical fields (e.g., damagedvehnum) were imputed using the median, while categorical variables (e.g., weather and trafficstate) were imputed using the mode. Since the missing data ratio for all fields was below 1%, no sample removal was necessary. Outliers in numerical variables were handled using the Interquartile Range (IQR) method. Values exceeding $1.5\times IQR$ from the first quartile $\left(Q1\right)$ or third quartile $\left(Q3\right)$ were capped at the 5th and 95th percentiles, respectively. These thresholds were calculated solely from the training set to prevent data leakage.

Regarding the unstructured text fields, a strict sanitization protocol was applied to simulate a realistic early-warning scenario and prevent temporal data leakage. While descriptive information regarding accident morphology was retained, all explicit timestamps (e.g., arrival or clearance times) and outcome-related statistics (e.g., casualty counts, direct financial loss figures) were masked prior to LLM processing. This ensures the model relies on semantic risk reasoning rather than calculating event duration or reading outcome labels. Additionally, basic text cleaning was performed to remove irrelevant special characters, but no manual tokenization or stop-word filtering was applied, as the LLM processes raw Chinese text using its internal tokenizer optimized for semantic reasoning.

2.3.2. Feature Construction

Based on the cleaned data, additional feature construction was performed to capture temporal and spatial accident patterns. From the alerttime attribute, multiple temporal features were derived, including the hour of occurrence (Hour), a binary indicator for weekends (Is_Weekend), and a categorical time-of-day representation (Time_Slot), partitioned into early morning [0–6), daytime [6–18), and nighttime [18–24). To capture regional risk aggregation, the Density-Based Spatial Clustering of Applications with Noise (DBSCAN) clustering algorithm ($eps=0.02,\mathrm{m}\mathrm{i}\mathrm{n}\_samples=10$) was applied to the longitude and latitude coordinates, generating a Spatial_Cluster_ID to identify high-incidence zones.

2.3.3. Feature Selection and Dataset Partitioning

Following feature construction, categorical variables—including vehicletypename, LaneName, and the newly generated Time_Slot and Spatial_Cluster_ID—were transformed into numerical representations using label encoding. Subsequently, Min–Max normalization was applied to all numerical features to scale them to the [0, 1] interval. Min–Max scaling was selected to maintain bounded feature ranges and ensure compatibility with continuous semantic scores generated by the LLM. Normalization parameters were learned exclusively from the training data and applied to the test data.

To reduce dimensionality and enhance model generalization, feature importance was evaluated using the Random Forest algorithm. The top 21 features were retained as the final feature set, comprising 12 structured attributes (A_Features) and 9 LLM-generated semantic features (B_Features), as illustrated in Figure 2. The cutoff was determined at the inflection point of the feature-importance curve and validated through cross-validation performance to avoid arbitrary thresholding. Finally, stratified sampling was employed to partition the dataset into an 80% training set (3210 samples) and a 20% test set (803 samples), ensuring consistent distribution of accident severity classes (minor, moderate, and severe) across both subsets. The detailed distribution is presented in Table 2.

Figure 2. Feature importance ranking comparison between LLM-generated semantic features and traditional structured attributes.

Table 2. Distribution of accident severity classes across the training and testing datasets.

Dataset	Total Samples	Minor Accidents (Count, %)	Moderate Accidents (Count, %)	Severe Accidents (Count, %)
Full Dataset	4013	(3893, 97.0%)	(73, 1.8%)	(47, 1.17%)
Training Set	3210	(3114, 97.0%)	(58, 1.8%)	(38, 1.18%)
Test Set	803	(779, 97.0%)	(15, 1.9%)	(9, 1.12%)

2.4. LLM-Based Semantic Feature Generation

The generation of LLM-based semantic features serves as the core component of the proposed framework. Its primary objective is to extract structured and interpretable high-level risk attributes from unstructured initial dispatch records to compensate for the semantic limitations inherent in conventional structured features. This process is implemented through a prompt-engineering paradigm, in which a multilayer prompt template is carefully designed to guide the model in analyzing raw police alert descriptions and producing proxy features in a standardized JSON format. The specific configuration and reasoning logic of this template are detailed in Table 3.

Table 3. Configuration of the Multilayer Prompt Template for Semantic Risk Reasoning.

Module	Description and Content
System Role	“Act as a senior traffic safety risk analyst. Analyze the provided unstructured accident description to infer potential risk levels and key features.”
Input Data	Alert Description
Reasoning Tasks (Output Fields)	1. Perceived Risk Index (0–5): Infer overall critical level based on accident morphology (e.g., fire = high risk).
	2. Injury Probability Score (0–5): Infer likelihood of injury from collision mechanism (e.g., high-speed impact).
	3. Vehicle Damage Extent (0–5): Infer physical damage from descriptive terms (e.g., “scrapped”, “deformation”).
	4. Traffic Impact Index (0–5): Assess impact on road capacity (e.g., “lane closure”).
	5. Accident Mechanism: Classify the physical nature (e.g., “Rear-end”, “Rollover”).
	6. Risk Contributing Factors: Infer latent causes (e.g., “Brake failure”, “Wet road”).
	7. Semantic Keywords: Extract high-weight risk descriptors.
Constraints	1. Return a strict JSON object.
	2. Prioritize semantic reasoning over keyword matching.

Unlike traditional keyword extraction, this stage utilizes a “Semantic Risk Reasoning” approach. Since the initial dispatch text often lacks explicit statistical outcomes (e.g., exact casualty counts), the prompt template defines a chain-of-thought [37] logic that guides the LLM to infer latent risk probabilities based on accident morphology (e.g., “rollover,” “fire,” “tunnel blockage”). The template explicitly defines the system role, domain-specific reasoning logic, output field types, and scoring criteria. As shown in Table 3, the generated semantic indicators include Perceived Risk Index, Injury Probability Score, and Vehicle Damage Extent, which serve as proxy variables for the actual accident severity.

For each accident record, the prompt template is combined with its corresponding sanitized AlertDescribe text and sent as a request to the DeepSeek-R1 [38] API. While the proposed semantic reasoning framework is adaptable to various LLMs (e.g., GPT-4, Llama 3), DeepSeek-R1 was selected for this study due to its specialized optimization for chain-of-thought reasoning tasks and its high proficiency in processing Chinese technical narratives. To enhance the reproducibility of the semantic features and mitigate the inherent stochasticity of Large Language Models, the API temperature parameter was set to 0.0 during the generation process. This configuration promotes near-deterministic decoding behavior by consistently selecting the most probable token at each step. Combined with a fixed prompt template and a constrained JSON-only output format, this setup substantially stabilizes the extracted risk scores across repeated runs for identical inputs. Leveraging its capabilities in contextual reasoning, the LLM analyzes key risk signals—such as “vehicle self-ignition” or “traffic interruption in tunnel”—and produces a structured JSON output strictly aligned with the predefined rules. The returned JSON content is subsequently parsed and converted into a feature matrix, forming a set of interpretable proxy features referred to as B_Features. These features provide direct evidence of incident risk patterns, enabling more accurate inference of accident severity even in the absence of structured data. Prior studies have demonstrated that such structured prompt engineering can significantly improve LLM performance on classification, reasoning, and feature extraction tasks [39].

The complete computational procedure for this feature generation process is formally summarized in Algorithm 1.

Algorithm 1. LLM-Driven Semantic Feature Generation
Input:	$\mathrm{Accident} \mathrm{records} D=\{d_1,d_2,\dots ,d_n\}$ $\mathrm{Prompt} \mathrm{template} P$ $\mathrm{Large} \mathrm{Language} \mathrm{Model} LLM$
Output:	$\mathrm{Semantic} \mathrm{feature} \mathrm{matrix} B$
1:	$\mathrm{Initialize} \mathrm{empty} \mathrm{feature} \mathrm{matrix} B$
2:	for each accident record di in D do
3:	Sanitize textual description (remove sensitive identifiers)
4:	$\mathrm{Construct} \mathrm{prompt} pi$$\leftarrow P\left(di\right)$
5:	$\mathrm{Query} LLM$ $\mathrm{with} pi$using temperature = 0.0 and JSON-only output constraint
6:	$\mathrm{Receive} \mathrm{structured} \mathrm{JSON} \mathrm{response} ri$
7:	$\mathrm{Parse} \mathrm{predefined} \mathrm{semantic} \mathrm{fields} \mathrm{from} ri$
8:	$\mathrm{Append} \mathrm{parsed} \mathrm{semantic} \mathrm{features} \mathrm{to} B$
9:	end for
10:	$\mathbf{Return} \mathrm{semantic} \mathrm{feature} \mathrm{matrix} B$

By integrating prompt engineering with accident narrative analysis and generating standardized JSON representations through API interaction, this stage establishes an interpretable, scalable, and efficient semantic feature extraction pipeline that serves as the foundation for subsequent feature fusion and predictive modeling.

2.5. Feature Fusion Strategies

To investigate how semantic features generated by the LLM contribute to and interact with traditional structured attributes in the accident severity prediction task, three fusion strategies were designed at different levels of integration.

2.5.1. Direct Concatenation Fusion (A + B_Features)

This strategy involves the direct horizontal concatenation of A_Features and B_Features to form a unified feature set. The concatenated dataset is partitioned into training and test sets and directly input into the prediction model. This serves as the baseline approach to evaluate the effectiveness of simple feature combination. Let the raw feature matrix $X_A\in {\mathbb{R}}^{m\times p}$ represent A_Features, where $m$ is the number of samples and $p$ is the dimension of A_Features. Similarly, let $X_B\in {\mathbb{R}}^{m\times q}$ represent B_Features, where $q$ is the dimension of B_Features. The fused feature set $X_{concat}$ is defined as:

$$X_{concat}=[X_A\mid X_B]\in {\mathbb{R}}^{m\times (p+q)}$$

(1)

where the symbol “$|$” denotes horizontal matrix concatenation.

This strategy serves as the baseline for evaluating whether structured and semantic features provide additive value when combined without interaction modeling.

2.5.2. Optimized Selection Fusion (Optimized A + B_Features)

Although direct feature concatenation is simple and effective, existing research suggests it may lead to dimensional redundancy and noise accumulation [33]. Therefore, we adopted a two-stage optimization strategy [34] aimed at retaining the most discriminative features while eliminating redundancy. The screening process consists of two steps. First, feature importance is evaluated based on the Random Forest algorithm to select a high-value feature subset $F_{high}$:

$$F_{high}=\{f_i\mid I(f_i)>\theta \}$$

(2)

where $I\left(f_i\right)$ represents the importance score of feature $f_i$, and $\theta$ is a preset threshold. Subsequently, features from the remaining set that have low redundancy with the high-value features and provide independent contributions are selected:

$$f_j\in F_{selected}⟺I(f_j)>{\theta }_1\wedge \underset{f_i\in F_{\mathit{high}}}{\mathit{max}}|\rho (f_i,f_j)|<{\theta }_2$$

(3)

where $\rho (f_i,f_j)$ denotes the correlation coefficient between features. Finally, the selected high-value features and the complementary features are concatenated to form a refined, low-redundancy optimized feature set, $X_{optimized}$.

This approach aims to retain only features with independent predictive value by jointly considering importance ranking and inter-feature correlation.

2.5.3. Model-Level Blending Fusion (Blended_Models)

Given the potential limitations of feature-level fusion, this study further explored a model-level blending strategy. Model blending has been proven in various fields to enhance classification stability and accuracy, particularly when weights are adjusted based on validation set performance [35]. This strategy leverages complementary learning behaviors between heterogeneous feature representations while avoiding noise propagation and computational burden of high dimensionality associated with feature-level fusion.

First, two independent base models (Base Model A and Base Model B, denoted as $M_{A }and{ M}_B$) are trained using A_Features and B_Features, respectively. This enables each model to focus on the intrinsic patterns of different feature sources. Second, probabilistic weighted averaging is applied during the inference phase. For a single test sample, the prediction probability vectors from both base models are obtained. Let $P_A\left(y\right|x)$ and $P_B\left(y\right|x)$ be the predicted probabilities for each accident severity class by models A and B, respectively. The final fused probability $P_{final}$ is calculated via weighted averaging:

$$P_{final}\left(y\right|x)=\alpha \cdot P_A(y\left|x\right)+(1-\alpha )\cdot P_B\left(y\right|x)$$

(4)

where $\alpha (0\le \alpha \le 1)$ is the weight assigned to $M_A$, which is optimized based on the performance on the validation set. The category with the highest probability in $P_{final}$ is selected as the final prediction result. This design enables independent representation learning and complementary generalization, particularly beneficial for minority class detection.

2.6. Experimental Setup

2.6.1. Implementation Details

All experiments were implemented using the Python 3.9 programming language and the XGBoost 2.0 framework. The training and inference processes were accelerated on a workstation equipped with an NVIDIA GeForce RTX 4090 GPU. To ensure the reproducibility of the results, the random seed was fixed at 42 for all data partitioning, model initialization, and sampling procedures.

2.6.2. Model Configuration and Hyperparameters

As the primary baseline and the core classifier for the ablation study, the XGBoost model was configured to optimize the multi-class objective function (multi:softprob) using the log-loss metric (mlogloss). Based on the grid search optimization performed during the cross-validation phase, the key hyperparameters were set as follows:

• Booster: Gradient Boosting Tree (gbtree) with histogram-based optimization (tree_method = ‘hist’) for efficient training on large-scale datasets.
• Estimators: The number of boosting rounds (n_estimators) was set to 100.
• Learning Rate: A learning rate (eta) of 0.1 was selected to balance convergence speed and generalization ability.
• Tree Depth: The maximum depth of trees (max_depth) was limited to 6 to prevent overfitting while capturing sufficient feature interactions.
• Regularization: Default regularization parameters were maintained to ensure model stability.

For the comparative models (LightGBM, Random Forest, and CatBoost), we followed a similar protocol, aligning their core parameters (e.g., number of trees and max depth) with XGBoost where applicable to ensure a fair comparison of their internal mechanisms.

2.7. Evaluation Metrics

To comprehensively evaluate the performance of the proposed prediction framework integrating LLM semantic features, a set of evaluation metrics tailored for multi-class and highly imbalanced data was employed. Given the skewed distribution of accident severity in the dataset (where minor accidents account for 97.0%, while moderate and severe accidents account for only 1.8% and 1.19%, respectively), relying solely on accuracy would yield a misleading assessment of the model’s true utility. Therefore, this study prioritized metrics capable of sensitively reflecting performance on minority classes, specifically:

• Accuracy

This measures the proportion of correctly predicted samples out of the total samples, representing the overall prediction precision. It is calculated as:

$$Accuracy={\displaystyle \frac{TP+TN}{TP+TN+FP+FN}}$$

(5)

where $TP$, $TN$, $FP$, and $FN$ represent True Positives, True Negatives, False Positives, and False Negatives, respectively. While providing a macroscopic overview, its reference value is limited in scenarios with severe class imbalance.

2.

Macro-F1 Score

Selected as one of the core metrics, the Macro-F1 score is calculated by computing the F1 score for each severity level independently (minor, moderate, severe) and then taking their arithmetic mean. Macro-averaging assigns equal weight to each class, effectively preventing the majority class from dominating the evaluation and sensitively reflecting performance on the scarce but critical “severe accident” class. The F1 score is the harmonic mean of Precision and Recall:

$$F1={\displaystyle \frac{2\times Precision\times Recall}{Precision+Recall}}$$

(6)

3.

Recall for Severe Accidents (Recall-Severe)

This study places special emphasis on the Recall of the “Severe Accident” class. This metric explicitly quantifies the risk of missed detections and is calculated as the ratio of correctly predicted severe accidents to the total number of actual severe accidents:

$${Recall}_{Severe}={\displaystyle \frac{T{P}_{Severe}}{T{P}_{Severe}+F{N}_{Severe}}}$$

(7)

This indicator holds the highest practical value, as in real-world traffic safety management, the cost of failing to predict (missing) a potential severe accident far exceeds that of a false alarm.

4.

Macro-AUPRC (Macro Average Precision-Recall Curve Area)

The Area Under the Precision-Recall Curve (AUPRC) is recognized as a more reliable metric than ROC-AUC for highly imbalanced datasets. We adopted a macro-averaging strategy, calculating the AUPRC for each class separately and then averaging them. This metric comprehensively assesses the model’s ability to identify positive instances (across all severity levels) under varying decision thresholds.

5.

Macro-Precision and Macro-Recall

To provide a more granular analysis of model performance across different categories, we also report Macro-Precision and Macro-Recall. Similar to the Macro-F1 score, these are derived by calculating the metric for each class and then computing the arithmetic mean, thereby treating all classes equally regardless of their sample size.

3. Results

This section presents the experimental results of the proposed traffic accident severity prediction framework. We first conducted a comprehensive ablation study based on the XGBoost classifier to identify the optimal feature fusion strategy. Subsequently, we extended this strategy to other mainstream ensemble learning models to verify its universality and robustness.

3.1. XGBoost-Based Fusion Strategy Exploration and Ablation Study

Given the extensive application and stability of XGBoost in structured data modeling [32], this study selected it as the primary classifier to dissect the contributions of different feature combinations. To systematically evaluate the independent effectiveness of LLM-generated features (B_Features) and their complementarity with traditional structured features (A_Features), a comprehensive ablation study was conducted. Additionally, to verify whether the performance improvement stems from LLM reasoning capabilities rather than mere semantic embeddings, we introduced a BERT-based feature set as a comparative baseline. We compared six feature configuration strategies:

• Baselines:
  • Structured Baseline (A_Features): The model was trained using raw structured features.
  • LLM Baseline (B_Features): The model was trained solely using LLM-generated semantic features.
  • NLP Baseline (A + BERT): Structured features were concatenated with Principal Component Analysis (PCA) -reduced embeddings extracted by bert-base-chinese to represent traditional semantic processing methods.
• Fusion Strategies: Three hierarchical fusion strategies (Direct Concatenation, Optimized Selection, and Model Blending) were implemented to investigate synergistic mechanisms between A_Features and B_Features.

All experiments employed 5-fold cross-validation with the SMOTEENN technique applied strictly within the training folds to prevent data leakage. The experimental results are presented in Table 4.

Table 4. Results of the ablation study on different feature configurations using the XGBoost classifier.

Feature Set	Macro-F1	Macro-Recall	Macro-Precision	AUPRC (Macro)	Recall (Severe)
A_Features	0.5874	0.6385	0.5529	0.4846	0.3333
B_Features	0.6234	0.7248	0.5966	0.6335	0.6667
A + B_Features	0.4632	0.4899	0.4469	0.4700	0.2222
A + BERT	0.5313	0.5126	0.5015	0.4092	0.2222
Optimized A + B	0.5432	0.5932	0.5232	0.5557	0.3333
Blended_Models	0.6356	0.7117	0.5958	0.6101	0.5556

Note: Bold values indicate the best performance in each column.

Table 4 delineates the quantitative performance of diverse feature configurations, revealing divergent efficacy across feature paradigms. The baseline model relying exclusively on structured attributes (A_Features) exhibited limited capability in discerning high-risk events, evidenced by a suboptimal severe accident recall of 0.3333. This underscores the inherent inadequacy of low-dimensional structured data in capturing the complex, tail-end dynamics of severe accidents.

Crucially, the introduction of traditional NLP embeddings (A + BERT) precipitated a notable decline in predictive fidelity rather than the anticipated enhancement. This configuration yielded a Macro-F1 of 0.5313 and a severe accident recall of 0.2222, falling below even the structured baseline. This counter-intuitive result suggests that high-dimensional dense vectors generated by BERT—which encode static contextual patterns without explicit causal reasoning—function principally as stochastic noise when fused with structured data, thereby diluting critical decision boundaries.

Conversely, the exclusive utilization of LLM-generated semantic proxies (B_Features) yielded a substantial performance leap, elevating the Macro-F1 to 0.6234 and the severe accident recall to 0.6667. This stark contrast with the BERT validates the hypothesis that explicit risk reasoning, rather than implicit semantic embedding, provides the essential discriminative leverage required for identifying minority class events.

Regarding fusion mechanisms, the naive concatenation of heterogeneous features (A + B_Features) resulted in marked performance degradation (Macro-F1: 0.4632), pointing to significant feature redundancy and interference. While the Optimized Selection strategy partially mitigated this issue, it failed to surpass the single-modality baselines. Ultimately, the Model-Level Blending strategy (Blended_Models) demonstrated superior robustness, achieving the highest Macro-F1 of 0.6356. By decoupling the learning processes of structured residuals and semantic reasoning, this strategy effectively synthesizes the complementary strengths of both modalities, ensuring robust identification of severe accidents (Recall: 0.5556) without compromising overall precision.

3.2. Robustness Analysis and Cross-Model Verification

Following the detailed ablation study on XGBoost, this section extends the evaluation to three other representative machine learning classifiers: LightGBM, Random Forest, and CatBoost. The primary objective is to verify the universality of the proposed LLM-driven semantic features (B_Features) and to investigate how different algorithmic architectures respond to the feature fusion strategies.

While the Model-Level Blending strategy proved optimal for XGBoost by leveraging residual information, different classifiers possess unique sensitivities to feature redundancy and noise. Therefore, rather than assuming a single optimal strategy, we reassessed the performance of the Structural Baseline (A_Features), the Pure Semantic Set (B_Features), and the Blended Strategy (Blended_Models) across these models. This comparative analysis aims to determine whether the predictive gain from LLM reasoning is model-agnostic and to identify the most effective integration method for each specific classifier. The experimental results are summarized in Table 5.

Table 5. Robustness evaluation of feature integration strategies across multiple machine learning classifiers.

Model	Feature Set	Macro-F1	Macro-Recall	Macro-Precision	AUPRC (Macro)	Recall (Severe)
LightGBM	A_Features	0.5581	0.6224	0.5204	0.4712	0.2222
	B_Features	0.6218	0.7605	0.5657	0.6158	0.7778
	Blended_Models	0.5794	0.6516	0.5361	0.6028	0.4444
RandomForest	A_Features	0.4823	0.5191	0.4611	0.4672	0.1111
	B_Features	0.6297	0.7112	0.5860	0.6036	0.5556
	Blended_Models	0.5539	0.6145	0.5175	0.5701	0.3333
CatBoost	A_Features	0.5702	0.6367	0.5303	0.4983	0.3333
	B_Features	0.6590	0.7487	0.6094	0.6454	0.6667
	Blended_Models	0.6254	0.7108	0.5731	0.6153	0.5556

Note: Bold values indicate the best performance in each column.

As evidenced in Table 5, the introduction of LLM-generated semantic features (B_Features) yielded consistent and significant performance improvements across all tested models compared to the baseline using only structured features (A_Features). In the Random Forest model, relying solely on structured features resulted in poor performance (Macro-F1: 0.4823, Severe Recall: 0.1111). However, employing B_Features alone propelled the Macro-F1 to 0.6297 and the severe accident recall to 0.5556, highlighting that semantic features provide high-quality information that compensates for the limitations of structured data in parallel decision-making mechanisms. Consistent with previous findings, the B_Features set also yielded the best performance for LightGBM and CatBoost. LightGBM achieved a severe accident recall of 0.7778, marking the highest value across all experimental configurations, while CatBoost reached its peak Macro-F1 of 0.6590 using the same feature set. These results demonstrate that the predictive power of LLM-based semantic extraction is robust and transferable across different classifiers.

Interestingly, unlike the XGBoost experiment (where Blended_Models performed best), the blending strategy did not outperform the pure B_Features configuration in LightGBM, Random Forest, or CatBoost. For instance, in LightGBM, the Blended_Models strategy resulted in a Macro-F1 of 0.5794, which is lower than that of B_Features (0.6218). A similar trend was observed in CatBoost, where the blended performance (Macro-F1 0.6254) trailed behind the pure semantic feature set (0.6590). This phenomenon suggests a “feature dilution” effect in these specific models. Since the A_Features (structured data) contain relatively high noise and weak discriminative power for minority classes (as seen in the low baselines), incorporating them via simple blending may interfere with the strong, clean signals provided by the B_Features. While XGBoost’s gradient boosting mechanism successfully leveraged the residual information from A_Features to complement B_Features, other models appeared to favor the “purity” of the high-dimensional semantic features.

4. Discussion

4.1. Superiority of Semantic Reasoning over Static Embeddings

The empirical results of this study provide compelling evidence for moving beyond traditional approaches to traffic accident prediction. While existing studies largely rely on structured data or shallow textual features, our experiments reveal that feature semantic density is the critical factor in prediction accuracy. A critical finding emerges from the comparison between the proposed LLM-based framework and the traditional NLP baseline (BERT). The BERT-based approach, which relies on static contextual embeddings, resulted in performance degradation (Macro-F1: 0.5313), failing to surpass even the structured baseline. This observation indicates that high-dimensional dense vectors, when lacking explicit causal structure, may contribute limited discriminative value in this task. In sharp contrast, the LLM-generated features (B_Features) achieved superior performance (Macro-F1: 0.6234). These results suggest that, in the context of accident severity prediction, semantic risk reasoning plays a more critical role than conventional textual representation based on static embeddings. By explicitly inferring risk proxies (e.g., Injury Probability) from sanitized narratives, LLMs capture the causal chain of accidents—such as inferring high risk from “tunnel smoke” even without explicit casualty numbers—capability that traditional embeddings lack.

4.2. The “Feature Dilution” Effect and Algorithmic Sensitivity

A compelling and counter-intuitive finding of this study is the non-monotonic behavior of feature fusion, termed the “feature dilution” effect. Contrary to the intuitive assumption that “more data yields better performance,” the introduction of structured features into LightGBM, Random Forest, and CatBoost paradoxically degraded performance compared to using LLM-derived features alone. Specifically, while the Level-wise growth strategy of XGBoost successfully leveraged structured attributes to capture residual variance, the Leaf-wise strategy of LightGBM appeared more sensitive to the information disparity between modalities.

This phenomenon can be attributed to the significant Signal-to-Noise Ratio (SNR) gap between the two feature sets. The LLM-generated semantic features (B_Features) exhibit extremely high information density, enabling LightGBM to establish distinct decision boundaries for minority classes (achieving a recall of 0.7778). In this context, the traditional structured features (A_Features), which contain implicit and noisy correlations, function as “informational pollutants” rather than supplements. When fused—whether through concatenation or blending—the lower-confidence probability distributions from the structured model dilute the high-confidence predictions derived from semantic reasoning. This suggests that, for high-capacity classifiers such as LightGBM, the quality and coherence of semantic reasoning may be more influential than the sheer quantity of structured features, highlighting the need for future research to develop adaptive gating mechanisms that can filter low-quality modalities before fusion.

4.3. Addressing the Long-Tail Distribution via Information Density

Regarding the inherent class imbalance in traffic data, this study offers a novel solution that serves as a robust alternative to traditional resampling. Severe accidents typically belong to the extreme minority class. Traditional methods (e.g., SMOTE) address this by synthesizing samples, which often introduces noise. Our results show that relying solely on LLM semantic features achieved a severe accident recall of up to 0.7778 without complex resampling. This indicates that the historical poor performance on minority classes was due to insufficient feature expressive power rather than sample scarcity. By mapping discrete accident descriptions into a continuous, high-dimensional semantic risk space, LLMs significantly enlarge the feature distance between “minor” and “severe” classes, thereby substantially reducing the learning difficulty for classifiers.

4.4. Generalizability and Regional Representativeness

Although the accident dataset used in this study originates from expressways in Yunnan Province, the proposed framework is not inherently region-specific. Yunnan’s expressway network is characterized by mountainous terrain, frequent tunnels, complex alignments, and diverse weather conditions, which give rise to a wide spectrum of accident scenarios. From this perspective, the dataset provides a challenging testbed rather than a simplified or homogeneous environment. More importantly, the core contribution of this study lies in the semantic risk reasoning paradigm rather than the statistical distribution of a specific region. The LLM-driven semantic features are derived from accident morphology and contextual descriptions (e.g., fire, rollover, traffic interruption), which constitute universal risk semantics shared across different road networks and regions. Therefore, the proposed framework can be readily transferred to other regions by adapting the prompt template to local reporting styles without modifying the underlying model structure.

4.5. Limitations and Future Directions

Despite the demonstrated effectiveness and promising generalizability discussed above, several limitations warrant attention. First, regarding data coverage, both temporal and spatial constraints exist. Temporally, the current dataset spans from January to August 2024. While this period captures critical seasonal variations—including the winter travel rush (Spring Festival) and the summer rainy season—it does not fully encompass the autumnal and late-year traffic characteristics. Extending the timeframe to a full annual cycle in future studies would further validate the model’s robustness against all seasonal periodicities. Spatially, while Yunnan’s complex terrain serves as a rigorous testbed, the validation is currently confined to a single province. Regional differences in police reporting conventions and traffic management codes may influence the textual expression of accident logs in other jurisdictions.

Second, the inference latency of large language models remains a constraint. The current API-based interaction restricts direct application in millisecond-level real-time warning systems where immediate low-latency response is required. Additionally, the prompt engineering currently relies on general reasoning capabilities, which may vary across different LLM architectures.

Future research should focus on three strategic directions:

• Model Distillation: Employing knowledge distillation to compress the heavy reasoning capabilities of LLMs into lightweight student networks (e.g., small-scale Transformers) to achieve real-time inference speeds suitable for edge deployment.
• Spatiotemporal Graph Integration: Integrating Graph Neural Networks (GNN) to combine the node-level semantic features extracted in this study with the topological structure of the road network. This would construct a next-generation “Neuro-Symbolic” model that synergizes macro-level spatiotemporal correlations with micro-level semantic risk reasoning.
• Cross-Regional Generalization: Extending the framework to multi-region and cross-country datasets to further evaluate its robustness against diverse reporting styles and validate the transferability of the semantic reasoning paradigm.

5. Conclusions

This study addresses the critical bottlenecks in traffic accident severity prediction—specifically the lack of semantic interpretation and the difficulty in identifying long-tail minority classes—by proposing a semantic feature enhancement framework driven by Large Language Models (LLMs). Unlike traditional methods that rely on structured data or static text embeddings, we devised a “Semantic Risk Reasoning” prompt paradigm to extract interpretable, high-dimensional risk proxies from unstructured dispatch records.

The core conclusions are threefold:

• Reasoning Superiority: We demonstrated that semantic reasoning outperforms semantic representation. The proposed LLM-based features significantly outperformed both the structured baseline and the BERT-based NLP baseline. Specifically, the pure semantic feature set achieved a Macro-F1 score of 0.6234, surpassing both the BERT-based baseline (0.5313) and the structured baseline (0.5874). This confirms that extracting explicit causal chains (e.g., inferring potential injury from accident morphology) is more effective for risk assessment than using “black-box” vector embeddings.
• Feature Dilution Discovery: We identified a “feature dilution” effect in multi-modal fusion, where adding low-quality structured data can degrade the performance of high-quality semantic features in certain classifiers (e.g., LightGBM). This challenges the conventional “all-in” data fusion strategy and highlights the need for classifier-aware feature selection.
• Imbalance Mitigation: We demonstrated that enhancing feature information density significantly improves the efficacy of standard balancing techniques. By combining LLM-driven reasoning with SMOTEENN, our framework achieved a remarkable recall rate for severe accidents. Notably, the LightGBM classifier utilizing LLM-derived features reached a severe accident recall of 77.8%, representing a substantial improvement over the 22.2% recall obtained by the structured baseline. This indicates that high-quality semantic features are a critical driver for minority class recognition.

In summary, this work provides a novel technical path for intelligent transportation systems, shifting the focus from “data quantity” to “semantic quality,” and establishes a theoretical foundation for interpretable, high-precision accident risk warning. Despite these contributions, we acknowledge certain operational limitations impeding immediate real-time deployment. Specifically, the current reliance on API-based interactions introduces challenges regarding inference latency, operational costs, and dependency on third-party services. Future work will prioritize model distillation to transition from cloud-based API calls to lightweight, edge-deployable solutions.