LSNet: Adaptive Latent Space Networks for Vulnerability Severity Assessment

Abstract

Due to the increasing harmfulness of software vulnerabilities, it is increasingly suggested to propose more efficient vulnerability assessment methods. However, existing methods mainly rely on manual updates and inefficient rule matching, and they struggle to capture potential correlations between vulnerabilities, thus resulting in issues such as strong subjectivity and low efficiency. To this end, a vulnerability severity assessment method named Latent Space Networks (LSNet) is proposed in this paper. Specifically, based on a clustering analysis in Common Vulnerability Scoring System (CVSS) metrics, we first exploit relations for CVSS metrics prediction and propose an adaptive transformer to extract vulnerability from both global semantic and local latent space features. Then, we utilize bidirectional encoding and token masking techniques to enhance the model’s understanding of vulnerability–location relationships, and combine the Transformer method with convolution to significantly improve the model’s ability to identify vulnerable text. Finally, extensive experiments conducted on the open vulnerability dataset and the CCF OSC2024 dataset demonstrate that LSNet is capable of extracting potential correlation features. Compared with baseline methods, including SVM, Transformer, TextCNN, BERT, DeBERTa, ALBERT, and RoBERTa, it exhibits higher accuracy and efficiency.

1. Introduction

As digitalization continues to advance, software vulnerabilities have become increasingly prominent. The International Business Machines Corporation (IBM) “2024 Cost of a Data Breach Report” [1] indicates that the average cost of data breaches, primarily caused by software vulnerabilities, has reached 4.88 million USD, resulting in a growing severity of economic losses. Pan et al. [2] propose that a strategy gaining traction is to detect new software vulnerabilities by analyzing related development activities, which facilitates timely alerts for open source software users. Consequently, in both enterprise economic activities and the process of users utilizing software, accurate and efficient vulnerability severity assessment is becoming increasingly significant. However, traditional vulnerability assessment methods predominantly rely on manual updates and inefficient rule-matching approaches [3]. These methods suffer from inherent limitations such as high subjectivity and inefficiency, rendering them inadequate for addressing increasingly complex cybersecurity threats.

In recent years, intelligent analytical approaches represented by machine learning algorithms have emerged as a research focus, including classification-based methods like Markov models [4], Support Vector Machines (SVM) [5], K-Nearest Neighbors (KNN) [6], and clustering-based methods like K-means cluster [7]. For example, Zheng et al. [6] proposed a KNN-based vulnerability exploitability assessment method. This approach pioneered a network environment-oriented evaluation framework and introduced a group expert–statistical dimensionality reduction paradigm for indicator selection, significantly enhancing assessment accuracy and effectiveness. However, most machine learning models depend on high-quality human-annotated data. For example, Microsoft proposes a “risk matrix” methodology to determine the severity levels of software vulnerabilities [8,9]. Vulnerability severity labeling necessitates both theoretical analysis and practical attack validation. Consequently, these methods exhibit constrained performance boundaries and limited generalizability when confronted with multi-source, heterogeneous, dynamically evolving vulnerabilities and complex threat scenarios. Additionally, they demand substantial domain-specific expertise. Currently, with the rise of deep learning technology, its capacity to autonomously learn deep-seated features of data through multilayered nonlinear networks offers a new paradigm for overcoming the performance constraints of traditional machine learning in vulnerability severity assessment. Compared to shallow machine learning models reliant on manual feature engineering, deep learning models like Convolutional Neural Networks (CNN) and their variants [10,11], as well as Recurrent Neural Networks (RNN) [12], significantly enhance automation and precision in complex scenarios.

Despite their success, the current paradigm faces critical limitations: it struggles to maintain high accuracy when processing highly specialized, domain-specific vulnerability descriptions characterized by intricate semantics, substantial information density, and urgent demands for rapid assessment [13]. Likewise, it is still not feasible to conduct vulnerability severity assessments in a fully automated and highly efficient method. Specifically, vulnerability data frequently suffer from noise interference and sparse critical features [14], while single deep learning models used for vulnerability assessment exhibit limitations in extracting composite features of critical vulnerabilities [15], as represented within Figure 1a.

To address these flaws, we propose a vulnerability classification model integrating clustering feature analysis and Transformer architecture by combining cluster analysis, the BERT pretrained model, Transformer structures, and TextCNN modules. The model decomposes vulnerability severity assessment into eight submodels, each responsible for classifying specific metrics within the Common Vulnerability Scoring System (CVSS) vector string. Initially, the pretrained BERT model generates word vectors from cleaned vulnerability text data. Subsequently, explicit and implicit textual features are fused through bidirectional encoding and token masking modules. The Transformer’s self-attention mechanism and convolution modules then extract global and local features from the vulnerability descriptions. Finally, the model jointly trains global and local latent space features alongside explicit and implicit information to enhance classification performance. The classification results are concatenated to form the CVSS vector string, from which the CVSS scores are calculated to determine vulnerability severity levels. This severity assessment serves as the basis for evaluating vulnerability threats, thereby enabling timely countermeasures. Its advantages are illustrated in Figure 1b. The main contributions of this paper are as follows:

(1)

We propose a novel vulnerability severity assessment model named LSNet, which handles longer textual information and extracts more effective latent associations features from vulnerability descriptions.

(2)

We perform feature extraction through clustering to enhance latent associations within the text. Subsequently, we propose bidirectional encoding and token masking to strengthen the model’s comprehension of textual semantics, and integrate the Transformer method with convolution to extract vulnerability text features, significantly enhancing the model’s capability to identify vulnerability texts.

(3)

Experiments conducted on NVD and CCF OSC2024, supplemented by vulnerability density fitting curves, demonstrate the highest accuracy of 97.1% and average mean squared error improvements of 11.16% in vulnerability severity assessment tasks compared to prior state-of-the-art vulnerability severity assessment methods and baseline models.

The rest of this paper is organized as follows. Section 2 discusses existing work on vulnerability severity assessment. Section 3 introduces the proposed model methodology, highlighting the innovation and rationale of this work. Section 4 discusses experiments and data analysis, presenting the dataset, testing environment, and comparative performance of model metrics, along with a detailed error analysis and ablation studies to comprehensively validate the method’s effectiveness. Section 5 concludes the paper and outlines future research directions. Similarly, we produce a table to summarize the key abbreviations used throughout the paper, as shown in

Table 1. Table of key abbreviations.

Completeness	Abbreviation
Latent Space Networks	LSNet
Common Vulnerability Scoring System	CVSS
Common Vulnerabilities and Exposures	CVE
National Vulnerability Database	NVD
Mean Squared Error	MSE

:

2. Related Work

In this section, we will first introduce the background of the creation of the CVSS, and then review the latest progress in the current state-of-the-art vulnerability severity assessment work and vulnerability severity assessment methods.

2.1. Common Vulnerability Scoring System

The Common Vulnerability Scoring System serves as a critical evaluation framework in information security, playing a pivotal role in quantifying vulnerability threats. Developed by the National Infrastructure Advisory Council (NIAC) and maintained by the Forum of Incident Response and Security Teams (FIRST), this system introduces standardized metrics to classify vulnerability severity, providing objective foundations for risk management [16]. It has gained widespread application in vulnerability prioritization, compliance auditing, and enterprise information security maintenance, establishing itself as the benchmark for modern vulnerability management ecosystems. The metrics comprise three key scores: the Base Score (BS), Exploitability Score (ES), and Impact Score (IS), and they can be used to quantify vulnerability severity [17]. These scores collectively measure: (1) vulnerability characteristics throughout the lifecycle; (2) domain-specific network security threats; and (3) potential impacts on end-users [3]. Derived from CVSS vector strings, these metrics are primarily determined by subject matter experts based on established evaluation criteria. The National Vulnerability Database (NVD) assigns standardized severity assessments for each vulnerability within its public repository, known as the Common Vulnerabilities and Exposures (CVE) list. This scoring methodology comprehensively considers exploitability, impact scope, attack complexity, and other factors, and is widely used and extensively adopted for vulnerability threat prioritization. While the system offers significant advantages in facilitating vulnerability prioritization, such as providing objective risk quantification, it has inherent limitations, including its limited adaptability to dynamic attack vectors, low automation level and excessive reliance on domain experts for manual judgment. Consequently, this paper concentrates on quantification gaps within specific vulnerability lifecycles, introduces an innovative analysis pathway incorporating comprehensive threat modeling to enhance precision in risk identification, and constructs a fully automated vulnerability assessment system based on CVSS3.1.

2.2. K-Means Clustering Based Vulnerability Severity Assessment

K-means clustering plays a crucial role in feature extraction. This algorithm partitions high-dimensional vulnerability data into K clusters by minimizing within-class variance, thereby revealing common patterns and shared characteristics of vulnerability exploits [18]. Applied in vulnerability detection, it offers a superior alternative to traditional rule-based feature analysis methods. K-means effectively captures feature data, such as code similarity, memory access patterns, and dynamic behaviors across vulnerabilities. This capability enhances the model’s comprehension of latent relationships between vulnerability descriptions and their corresponding threat levels [19,20]. The current framework of vulnerability clustering approaches primarily focuses on analyzing latent correlations among vulnerabilities, thereby neglecting essential dimensions required for comprehensive severity assessment, leading to suboptimal classification outcomes. Thus, this study bridges this gap by leveraging transformer technology to integrate clustering features with severity analysis mechanisms.

2.3. Transformer Model

Vaswani et al. [21] introduced a Transformer encoder–decoder architecture based on the self-attention mechanism, offering a novel solution for vulnerability severity assessment. This framework abandons the sequence modeling paradigm of traditional recurrent neural networks (RNNs), employing a structure solely based on the attention mechanism to model global dependencies in input sequences [22]. The model significantly enhances parallel processing capabilities and long-range dependency capture through multi-head attention and feed-forward neural networks. For vulnerability severity assessment tasks, the Transformer encoder–decoder architecture effectively handles long-distance dependencies and complex semantic structures within vulnerability description texts. The self-attention mechanism enables the model to dynamically focus on critical vulnerability features, while multi-head attention concurrently captures global contextual representations, thereby extracting holistic characteristics of vulnerability texts and identifying intricate patterns within them [23,24]. Previous Transformer-based vulnerability severity assessment models still fail to integrate underlying correlations because the models ineffectively capturing latent dependencies between vulnerabilities. Building on that basis, this paper integrates feature biases derived from K-Means clustering to optimize the vulnerability classification model. By incorporating clustering patterns into the Transformer framework, the approach enables the model to assimilate both explicit dependency relationships and latent cluster characteristics, thereby achieving enhanced assessment accuracy for vulnerability severity levels through multi-dimensional feature fusion. This integration strategy significantly advances beyond conventional single-model solutions.

3. Method

In this paper, we propose a vulnerability severity assessment model integrating clustering feature analysis and the Transformer method, which comprises four hierarchical layers, depicted in Figure 2. First, the word-embedding layer processes vulnerability information text through data cleaning and lexical preprocessing, converting tokens into node vectors. This layer incorporates bidirectional text encoding and token masking mechanisms to enhance the positional awareness of Common Vulnerability Scoring System vector strings [25]. Secondly, the TextCNN-based [26] feature extraction layer captures local textual patterns while integrating clustered features. Third, the Transformer-TextCNN feature fusion layer amalgamates vulnerability characteristics extracted via clustering, further strengthening the model’s positional sensitivity to vector strings. Finally, the fully connected output classification layer synthesizes distributed features, generates one-hot probability distributions, and ultimately produces predicted classification.

3.1. Clustering Feature Analysis

In this paper, we propose clustering-based data mining analysis as its foundational motivation, uncovering a novel method for predicting CVSS scores. It utilizes the results of clustering feature analysis as feature weights during model training.

Given the complexity of raw textual data, dimensionality reduction becomes essential for processing the extracted features from Common Vulnerabilities and Exposures vulnerability text. Principal Component Analysis (PCA) [27] serves as the dimensionality reduction technique, with empirical findings indicating optimal performance at two dimensions. This approach sufficiently preserves feature integrity while significantly streamlining data complexity [28]. Subsequent standardization processing enables the revelation of latent vulnerability behavioral patterns within the data. The methodology involves in-depth analysis of textual information through the CVSS framework to examine inter-component relationships across vulnerability elements. K-Means clustering then excavates underlying threat patterns from this analysis. Simultaneously, a postulated Gaussian Mixture Model investigates Gaussian distributions across vulnerability datasets, simulating vulnerability propagation dynamics throughout their lifecycle evolution [29].

The desired outcome for each cluster is maximized intra-cluster similarity and minimized inter-cluster similarity. To quantify cluster separability, this methodology introduces Minkowski distance measurement ${Dk}_m\left(x_i,x_j\right)$ and cosine similarity assessment $cos\left(\theta \right)$ as dual quantitative metrics.

$${Dk}_m\left(x_i,x_j\right)={\left({{\sum }_{\mu =1}^n\left|x_{i\mu }-x_{j\mu }\right|}^p\right)}^{{\displaystyle \frac{1}{p}}}$$

(1)

$$\mathit{cos}\left(\theta \right)={\displaystyle \frac{{\sum }_{i=1}^n\left(x_i\times y_i\right)}{\sqrt{{\sum }_i^n{\left(x_i\right)}^2}\times \sqrt{{\sum }_i^n{\left(y_i\right)}^2}}}$$

(2)

In the formula above, $x_i$ denotes the lateral separation distance between clusters, $k$ is the defined dimensionality and $p$ is the order of distance, while $x_i\mathrm{a}\mathrm{n}\mathrm{d}{y}_i$ represent the horizontal and vertical separation distance.

By integrating the following three comparative methods and applying the elbow criterion, it is possible to determine an effective combination of dimensionality reduction parameters and cluster count [30]:

(1) Sum of Squared Errors Criterion: A smaller sum of squared errors indicates better clustering performance, which measures the predictive accuracy of a model by calculating the sum or average of squared differences between predicted and actual values, as follows:

$$SSE=\sum _{i=1}^k\sum _{\rho \in X_i}{\left|\left|\rho -{\partial }_i\right|\right|}^2$$

(3)

where $\rho$ denotes observed values and ${\partial }_i$ represents predicted values, followed by summation over all samples. This formula quantifies the discrepancy between predicted and observed values.

(2) Silhouette Coefficient: This metric evaluates the impact of different algorithms or algorithmic configurations on clustering results using identical raw data and quantifies how well each data point fits into its assigned cluster relative to other clusters. Normalized within [−1, 1], values closer to 1 signify superior intra-cluster cohesion and inter-cluster separation.

$${SC}_i={\displaystyle \frac{{\delta }_i-{\sigma }_i}{\max \left\{{\delta }_i,{\sigma }_i\right\}}}$$

(4)

$${\sigma }_i={\displaystyle \frac{1}{n-1}}\sum _{\phi !=\tau }^{n}distance\left(\phi ,\tau \right)$$

(5)

Specifically, ${\delta }_i$ measures the cohesion of a sample point, where $\tau$ denotes other samples within the same cluster as sample $\phi$, and distance computes the distance of $(\phi ,\tau )$. ${\sigma }_i$ is computed similarly to ${\delta }_i$, but requires iterating through other clusters to obtain multiple values, with the minimum selected as the final result. A smaller ${\sigma }_i$ indicates tighter cluster cohesion.

(3) Calinski–Harabasz Index: Defined as the ratio of between-cluster dispersion to within-cluster dispersion and quantifying the compactness and separation of clusters by calculating the ratio of between-cluster variance to within-cluster variance, where higher values denote better clustering outcomes.

$$CH=\hspace{0.33em}{\displaystyle \frac{{B_k}^T}{{W_k}^T}}\hspace{0.33em}\times \hspace{0.33em}{\displaystyle \frac{N-k}{k-1}}$$

(6)

where $CH$ represents the Calinski–Harabasz Index. $B_k$ denotes the between-cluster scatter matrix, reflecting dispersion among cluster centroids, while $W_k$ is the within-cluster scatter matrix, indicating dispersion among samples within the same cluster. $N$ indicates the total number of vulnerability data points, and k represents the number of clusters.

However, considering practical constraints, excessively low dimensionality reduction may lead to an insufficient data volume in some clusters. Therefore, we select a configuration of 2-dimensional reduction with 4 clusters, which significantly mitigates the sample distribution imbalance issue [31]. The resulting clustering outcomes are visually presented in Figure 3, where the x-axis represents the first principal component, and the y-axis represents the second principal component. Principal components are linear combinations of all original features. The first principal component (PC1) indicates the direction that maximizes data variance, while the second principal component (PC2) is orthogonal to PC1 and explains the maximum remaining variance.

In this paper, we propose heatmap visualization to analyze the clustered data, examining the relationships and magnitudes of influence between individual metrics in CVSS vector strings and multidimensional CVSS scores. The heatmap illustrating the clustering analysis is depicted in Figure 4.

Consequently, the vulnerability severity assessment method proposed in this paper subdivides CVSS score prediction into eight distinct subproblems, each corresponding to individual metrics within the CVSS vector string [32]. By predicting outcomes for each metric, the approach synthesizes the predicted CVSS vector string, with the final CVSS scores thus calculated. This research designs a BERT-based deep learning model for CVSS scores prediction. The model separately trains and predicts each metric within the CVSS vector string, aggregates results, and consequently computes the CVSS scores for a given vulnerability instance.

3.2. Word Embedding

In this subsection, we elaborate on the word-embedding approach, covering data cleaning, initialization operations, encoding techniques, and the token-masking module to introduce the input data processing for LSNet’s training.

3.2.1. Data Cleansing

Before model training, data cleansing is essential to streamline the dataset and align it with task requirements. This paper adopts an approach to discard irrelevant columns such as “CVE-ID”, “Issue_Url_old”, “Issue_Url”, and “Repo_new” from the text information. Although potentially useful in other contexts, these features generally contribute minimally to vulnerability analyses or vulnerability correlation training. Consequently, retaining the vulnerability description text, CVSS vector strings, CVSS scores, and threat severity levels proves sufficient. For disordered vulnerability labels and irrelevant gibberish (e.g., “CVE_ID”:”CVE-2018-20847,” “description”:”Heap buffer overflow... NUMBERTAG ERRORTAG... URLTAG... NUMBERTAG”), regular expressions filter out erroneous terms and meaningless noise. Without this cleaning step, such artifacts would cause the model to fit noise and converge to a non-optimal solution.

3.2.2. Data Preprocessing

Preprocessing serves as a critical phase in natural language processing, transforming raw text into structured and standardized formats [33]. The subsequent clustering analysis suggested visualizing the processed data through heatmaps to examine relationships between individual CVSS v3.1 vector metrics and multi-dimensional CVSS severity scores. These visualizations reveal the contribution weights of each vector metric toward the final CVSS scores [34], reflecting their criticality in severity assessment. This preprocessing methodology enhances semantic comprehension for subsequent analytical models. Word preprocessing in our framework involves two sequential operations:

(1)

Stemming reduces words to their root forms, a technique widely applied in linguistic preprocessing. For vulnerability severity assessment, stemming extracts key terminology from vulnerability descriptions, simplifying feature expressions while boosting model convergence efficiency.

(2)

Lemmatization converts words to their canonical forms based on grammatical context. Unlike stemming, lemmatization accounts for syntactic roles, ensuring morphological standardization. In CVSS severity analysis, lemmatization strengthens the model recognition of polymorphic vulnerability-related terms. For instance, the words “attack” and “attacked” were grouped into identical lexical entries, minimizing interpretation ambiguities.

Our experiments use the Natural Language Toolkit (NLTK) for text preprocessing. Input vulnerability descriptions are first tokenized, then processed through NLTK’s Porter stemmer and WordNet lemmatizer modules. This pipeline standardizes lexical variations while preserving critical semantic indicators essential for transformer-based vulnerability classification.

3.2.3. Bidirectional Information Encoding

For vulnerability severity assessment tasks, transforming them into metric classification tasks for CVSS vector strings reveals that the sequence and relative position information of feature metrics play a decisive role in accurately parsing and determining vulnerability characteristics [35]. Given that encoding itself does not involve directionality, while bidirectional information is captured through model architecture, different orders of occurrence within the sequence often represent vastly different vulnerability configurations and severity implications. Thus, the proposed model first integrates an explicit bidirectional information encoding module, synergistically coupled with an implicit token masking mechanism at the subsequent layer. During word embedding of CVSS vector strings, the foundational Transformer model typically presents learnable absolute positional encoding to encode positional indices of each token in the sequence [36]. The resulting embedding vector matrix $X_{\mathit{BERT}}=\{X_1,X_1,\dots ,X_n\}$ contains position features implicitly acquired through internal model processing, primarily relying on absolute token indices [21,25]. However, this mechanism places greater emphasis on fixed sequential order, struggling to adequately model complex and critical functional dependencies and relative distance relationships among vulnerability metrics within vector strings, such as combinatorial effects between adjacent metrics or tight interconnections within specific metric groups.

To more effectively capture such inter-metric relative positional semantics embedded in CVSS vector strings, a bidirectional information encoding approach distinct from absolute positional encoding exclusively focused on fixed sequences is proposed. Beyond mere sequential order considerations, bidirectional information encoding can perceive forward and backward relative positional offsets between tokens, for example, the offset of metric AC relative to AV or C relative to I, while also offering inherent continuity and smoothness advantages [37]. Leveraging the periodic nature of sine and cosine functions, this encoding strategy enables the model to more robustly learn relative positional relationship features between any two metrics across varying distance scales, thereby significantly enhancing the comprehension of structured semantics in CVSS vector strings and improving classification accuracy for the metrics.

3.2.4. Token Masking

Token masking segments the string into minimal tokens to accurately localize specific indicator values for masking operations, thereby preventing the model from disrupting logical associations between labels and values and enabling the model to learn finer positional relationships in text [38]. This model utilizes the masked language modeling capabilities of BERT to implement token masking on CVSS vector strings. For the input CVSS vector string label $[y_1,y_2,\dots ,y_n]$, we randomly mask and replace certain tokens of each indicator label with the special token [MASK] [25,39]. The model then predicts and recovers these masked tokens based on contextual information, leveraging vulnerability description text as input word vectors and the original CVSS vector string as the learning target label. This approach outputs classification results for each indicator in the CVSS vector string. The specific design of token masking is detailed in Figure 5.

3.3. Transformer Adaptive Latent Space Feature Fusion

We propose LSNet, which integrates the self-attention mechanism of the Transformer method to capture global feature relationships within sequences, subsequently incorporating residual networks, layer normalization, and feed-forward networks to enhance model performance and training efficiency. For the transformer-encoded vulnerability text data $X_{\mathit{BERT}}=\{X_1,X_2,\dots ,X_n\}$, the hidden states of the input sequence $H\in R^{n\times d}$ (where $n$ represents the sequence length and $d$ denotes the hidden layer dimension) undergo linear transformations to generate query ($Q$), key ($K$), and value ($V$) matrices as follows:

$$\begin{array}{c}Q=H{W}^Q\\ K=H{W}^K\\ V=H{W}^V\end{array}$$

(7)

where $Q$, $K$, and $V$, respectively, represent the query, key, and value matrices for the head, and $W^Q$, $W^K$, $W^V$ denote learnable weight matrices. We then use the following equation:

$$Attention\left(Q,\hspace{0.33em}K,\hspace{0.33em}V\right)=Softmax\left({\displaystyle \frac{Q{K}^T}{\sqrt{d_k}}}\right)V$$

(8)

where $d_k$ signifies the dimension of the key vector. Subsequently, we concatenate the attention scores $Attentio{n}_i$, obtained from h attention heads, and then multiply this concatenated result by the output projection weight matrix to obtain matrix $Z$ [21].

Then, TextCNN is used for feature extraction, where a textual description of vulnerability $[x_1,x_2,\dots ,x_n]$ is first passed through the convolutional layer to filter individual word vectors. The resulting vectors are then utilized as inputs to TextCNN. Using four adaptively sized convolutional kernels, the model extracts feature maps by performing convolution operations on the text sequence [26]. The resulting feature map sequences, denoted as ${Feature}_i$, incorporate the clustering vector association weights ${Clusterb}_i$ obtained from Section 3.1 as convolutional bias, as illustrated below:

$$Featur{e}_i=Conv\left(Z,W^V\right)+Cluster{b}_i$$

(9)

$$X_{TextCNN}=Con{v}_1+Con{v}_2+Con{v}_3+Con{v}_4$$

(10)

$$Cluster{b}_i=\left({\alpha Dk}_m\left(x_i,x_j\right)+\beta \mathit{cos}\left(\theta \right)\right)\times {SC}_i$$

(11)

where $+$ denotes vertical stacking, while $Conv$ represents the convolution operation, $\alpha ,\beta >0$, determined by the optimizer. Leveraging TextCNN, we propose multi-channel, multi-kernel convolution operations on feature representation vectors generated by the Transformer encoder. This process extracts local commonality features across varying granularities of textual words and syntactic structures. Subsequently, maximum pooling is applied to achieve feature dimensionality reduction.

The pooled feature vectors are fed into the representation integration layer for classification operations and undergo nonlinear transformation via an adaptive activation function. The concatenated feature vector $V=[v_1,v_2,\dots ,v_n]$ is computed with the fully connected layer’s weight matrix $W_v$ and bias term $b_v$.

$$Z=W_{v}V+b_v$$

(12)

$$H_v=ReLu\left(Z\right)$$

(13)

where $W_{v}V$ denotes the weight matrix of the representation integration layer, and $b_v$ represents the bias term that dynamically adjusts via adaptive attention mechanisms. The resulting $H_v$ constitutes the nonlinearly transformed string feature vector.

The feature distribution $H_v$, after max pooling in the feature fusion layer, is input into a fully connected layer for classification, transforming it into a feature probability vector $Z_{Decoder}=[Z_1,Z_2,\dots ,Z_n]$. This vector undergoes normalization via the softmax function, enabling multi-class classification of each metric within the CVSS vector string to obtain their corresponding one-hot probabilities:

$$P_{o-h}=Softmax\left(Z_{Decoder}\right)$$

(14)

In this process, LSNet outputs the label value for each CVSS vector string. It applies the softmax function to determine the class with the highest probability, assigns a value of 1 to that class, and sets all other classes to 0, thereby obtaining the definitive label value for each CVSS vector string.

During experimental training, the model uses cross entropy as the loss function to quantify the discrepancy between predictions and ground truth labels. The Adam optimizer utilizes backpropagation to iteratively refine training parameters. This process progressively reduces classification loss while enhancing the model’s classification accuracy. The specific calculation for the cross entropy loss function is as follows:

$${Loss}_{c-e}=-\sum _{i=1}^L{G}_i\log \left(P_{o-h}\right)$$

(15)

where $G_i$ represents the ground truth (GT) label, $L$ denotes the total number of entries in the CVSS vector string, and $P_{o-h}$ represents the model-predicted probabilities.

4. Experiment and Analysis

In this section, we comprehensively outline the overall framework for experimental design and result analysis. We first introduce the experimental setup and basic procedures. By comparing the performance of different models, we systematically assess the effectiveness of various methods. Furthermore, we combine multiple metrics and error analysis of prediction results to thoroughly investigate model behavior and limitations. Finally, we employ ablation experiments to validate the role of key components and their contributions to overall performance.

4.1. Dataset

To comprehensively validate the effectiveness of our proposed model for vulnerability severity assessment, we collected publicly available vulnerability data from the NVD (https://nvd.nist.gov/, accessed on 27 July 2025), spanning January 2018 to December 2024. After data preprocessing to remove entries with missing critical information, we obtained 126,607 valid entries. We randomly partitioned this corpus into training, validation, and test sets to form a large-scale dataset denoted as NVD throughout this paper. Additionally, we incorporated the vulnerability severity assessment dataset from the 7th China Computer Federation Open Source Competition (https://www.gitlink.org.cn/competitions/track4_vulnerability, accessed on 27 July 2025) as a small-scale dataset, referred to as the CCF OSC2024 in this paper (

Table 2. Statistics of datasets.

Dataset		Total
NVD	Training Set	101,285
	Validation Set	12,661
	Testing Set	12,661
CCF OSC2024	Training Set	5624
	Validation Set	710
	Testing Set	710

).

4.2. Experimental Setup

In this paper, all experiments employed identical environmental configurations. The operating system was Ubuntu 22.04 LTS, with an NVIDIA GeForce RTX 4090 GPU and Python 3.9 as the programming language. Deep learning models were built using PyTorch 2.5.0. Regarding hyperparameter selection, we refer to the original papers of BERT and TextCNN, employing the most commonly adopted initial learning rate, convolution kernel count and dimensions, convolution fusion layer parameters that demonstrated optimal performance for the classification task. The Adam optimizer dynamically adjusts the learning rate during training, with specific configuration details provided in

Table 3. Model parameter settings.

Parameter	Value
Word vector dimensionality	768
Maximum text length	256
Epochs	5
Learning rate	0.000001
Loss function	CrossEntropy
Optimizer	Adam
Number of TextCNN convolutional kernels	4
TextCNN convolutional kernel size	3, 4, 5
Number of TextCNN convolutional fusion layers	12

.

4.3. Evaluation Metrics

In this paper, we introduce quantitative methods to evaluate model performance and vulnerability severity assessment outcomes. For the classification results of CVSS vector string metrics generated by the model, samples are categorized into four types based on their ground truth labels and model predictions: correctly classified positive samples are true positives (TP), incorrectly classified positive samples are false positives (FP), correctly classified negative samples are true negatives (TN), and incorrectly classified negative samples are false negatives (FN). Based on these categorizations, the model’s accuracy (Acc), Precision, and F1 Score serve as evaluation metrics. The corresponding formulas are as follows:

$$Acc={\displaystyle \frac{TP+TN}{TP+FP+TN+FN}}$$

(16)

$$Precision={\displaystyle \frac{TP}{TP+FP}}$$

(17)

$$F1\hspace{0.33em}Score=2\times {\displaystyle \frac{Precision\times Recall}{Precision+Recall}}$$

(18)

Within the CVSS, the classification results of CVSS vector string metrics generated by our model undergo standardization through a computational formula, yielding normalized values within the [0, 10] interval. Consequently, this paper additionally adopts mean squared error (MSE) as a performance evaluation metric for the proposed vulnerability severity assessment method, thereby demonstrating its effectiveness. MSE represents the average of the squared differences between true values and predicted values, serving to quantify the deviation between observations derived from our vulnerability severity assessment approach and ground truth. The calculation formula is as follows:

$$MSE\hspace{0.33em}=\hspace{0.33em}{\displaystyle \frac{1}{N}}\sum _{t=1}^N{\left(G{T}_t-Pre{d}_t\right)}^2$$

(19)

where $G{T}_t$ is the true label value and $Pre{d}_t$ is the predicted label value.

4.4. Model Performance Comparision

To validate the effectiveness of the proposed model, we conducted comparative performance experiments against seven baseline models using two datasets. Evaluation metrics included accuracy (Acc), precision, F1 Score, and the mean squared error (MSE) of CVSS scores calculated by combining and aggregating model outputs. Experiments evaluated the aforementioned models and the proposed method using both standard datasets, with detailed performance comparison outcomes demonstrated in

Table 4. The performance comparison between the proposed model and other baseline models on ACC.

Model	Acc
	AV	AC	PR	UI	S	C	I	A
SVM	80.9	86.7	60.1	81.3	85.7	69.1	68.9	72.6
Transformer	86.9	88.2	62.7	89.8	88.3	80.5	72.4	80.3
BERT	84.3	96.5	61.6	88.1	90.6	78.3	73.7	76.3
BERT-TextCNN	88.5	96.0	77.4	84.3	93.8	82.7	82.2	84.9
DeBERTa	85.1	96.8	68.8	83.7	89.1	73.5	71.5	76.8
ALBERT	87.7	90.2	68.3	90.2	94.7	78.6	75.5	80.4
RoBERTa	85.6	96.4	68.3	84.2	92.6	71.8	71.3	79.5
LSNet (Ours)	91.3	97.1	80.4	92.5	96.0	86.2	86.8	88.5

,

Table 5. The performance comparison between the proposed model and other baseline models on precision.

Model	Precision
	AV	AC	PR	UI	S	C	I	A
SVM	76.2	82.8	65.0	80.3	80.7	67.4	63.9	70.4
Transformer	72.7	82.5	69.8	86.7	82.1	75.0	76.8	80.3
BERT	86.9	96.4	70.3	89.5	91.0	79.8	75.3	81.5
BERT-TextCNN	89.5	96.7	82.9	90.6	93.3	88.1	80.2	87.6
DeBERTa	83.0	96.5	69.1	84.3	89.1	76.4	73.2	76.8
ALBERT	87.4	94.0	70.9	90.2	94.8	81.1	75.9	82.2
RoBERTa	83.2	96.1	68.7	92.3	85.2	73.6	73.4	78.9
LSNet (Ours)	93.7	98.2	83.3	92.0	97.1	87.4	87.3	90.6

and

Table 6. The performance comparison between the proposed model and other baseline models on the F1 Score.

Model	F1 Score
	AV	AC	PR	UI	S	C	I	A
SVM	0.74	0.83	0.60	0.79	0.85	0.71	0.80	0.74
Transformer	0.70	0.87	0.64	0.88	0.86	0.81	0.83	0.77
BERT	0.87	0.95	0.63	0.87	0.89	0.77	0.79	0.76
BERT-TextCNN	0.89	0.92	0.71	0.88	0.93	0.78	0.85	0.80
DeBERTa	0.83	0.96	0.65	0.83	0.88	0.73	0.71	0.76
ALBERT	0.87	0.92	0.64	0.90	0.95	0.79	0.76	0.80
RoBERTa	0.84	0.95	0.64	0.83	0.92	0.72	0.71	0.79
LSNet (Ours)	0.92	0.98	0.81	0.92	0.96	0.86	0.87	0.89

.

The proposed model demonstrates smaller errors in CVSS score prediction with a more stable error distribution, evidencing its high prediction accuracy and strong stability. However, opportunities for improvement remain regarding the impact score, primarily due to the Confidentiality (C), Integrity (I), and Availability (A) indicators within the CVSS vector strings. As Table 3 indicates, the model exhibits suboptimal prediction accuracy for these three specific metrics.

4.5. Analysis of Experimental Results

The model output results are merged and concatenated into CVSS vector strings, which are then input into the CVSS calculator toolkit to consequently compute all CVSS scores. Based on these results, we conduct error analysis and generate density fitting plots, integrating ablation experiments to demonstrate the effectiveness of our model.

4.5.1. Analysis of Comparative Experiments on Baseline Models

To validate the effectiveness of our model, performance evaluation and experimental error analysis were conducted for three distinct score categories. The selected performance metrics included accuracy (Acc), precision, and F1 Score, while mean squared error (MSE) was utilized for error analysis. Baseline models comprised the following:

(1)

SVM [5]: This approach utilized support vector machines to extract text features, compute text similarity, and classify CVSS vector strings.

(2)

BERT [25]: Leveraging the BERT pretrained model to dynamically generate feature vectors, it measured text similarity through cosine similarity between sentence embeddings for CVSS vector string classification.

(3)

BERT-TextCNN [40]: Combining BERT embeddings with TextCNN convolutional layers to extract local features, this method generated sentence vectors via pooling and classified CVSS strings using cosine similarity.

(4)

DeBERTa [41]: Utilizing the DeBERTa pretrained model for dynamic feature representation, it determined text similarity through the cosine distance between sentence vectors to classify CVSS strings.

(5)

ALBERT [42]: Utilizing the ALBERT pretrained model to dynamically generate sentence embeddings, it classified CVSS vector strings by calculating the cosine similarity between vectors.

(6)

RoBERTa [43]: Adopting the RoBERTa pretrained model for dynamic sentence representation, it classified CVSS strings through cosine similarity comparisons.

The error comparison of baseline models is presented in

Table 7. The performance comparison between the proposed model and other baseline models on MSE.

Model	NVD			CCF OSC2024
	BS	ES	IS	BS	ES	IS
SVM	5.46	2.24	15.8	9.13	4.58	28.0
Transformer	3.53	1.76	8.39	7.40	3.67	20.7
BERT	3.21	0.79	7.75	7.89	2.14	19.6
BERT-TextCNN	2.42	0.70	9.06	6.01	1.29	20.1
DeBERTa	3.89	1.10	12.4	6.54	3.11	25.2
ALBERT	3.88	0.96	8.32	6.93	1.85	22.5
RoBERTa	3.68	1.13	10.7	6.46	1.65	25.9
LSNet (Ours)	2.15	0.61	8.97	5.59	0.93	21.4

.

As evidenced in Table 7, models based on BERT and its variants, including BERT-TextCNN, DeBERTa, ALBERT, and RoBERTa, demonstrate superior classification performance over traditional SVM methods. This outcome confirms that BERT’s pre-trained embeddings capture rich textual features crucial for enhancing classification efficacy, thus justifying BERT as the selected word embedding approach. The BERT-TextCNN hybrid reduces MSE for base CVSS scores by 24.6% and 23.8% across two datasets compared to standalone BERT, highlighting the TextCNN module’s role in strengthening local feature extraction and interconnections.

In vulnerability severity assessment, particularly for multi-class predictions, the proposed model delivers significant improvements in accuracy, F1 Score, and precision relative to baseline models. Figure 6 and Figure 7 visually compare its MSE for base CVSS scores against RoBERTa, a representative baseline.

The results in Table 6 confirm the proposed model’s superior performance across both vulnerability text datasets. These gains stem from its integrated architecture, combining clustering feature analysis, Transformer mechanisms, and TextCNN modules. By effectively fusing bidirectional encoding and token masking, the model enhances positional awareness. The Transformer’s self-attention captures global dependencies, while TextCNN extracts local patterns, collectively strengthening generalization. This optimized design yields state-of-the-art results in vulnerability severity assessment.

In CVSS, vulnerabilities with base scores in [7, 10] are judged as high-risk vulnerabilities. The model proposed has a low square error in judging the severity level of high-risk vulnerabilities, and the prediction effect is excellent, as shown in Figure 8. This shows that the model proposed can achieve a good assessment effect on high-risk vulnerabilities without any manual intervention, significantly improving the speed and accuracy of security detection.

4.5.2. Fitted Density Plot

In this paper, we plot the real year density fitting curve and the predicted year CVSS scores density fitting curve to label vulnerability severity levels and compare the differences. Based on the density fitting graph of the experimental data, we observe that the predicted curve shows general consistency with the real curve, following the same trend. The fluctuations in the predicted curve maintain good synchronization with those in the real curve, and the error range remains within acceptable limits. Experimental results demonstrate that our model not only reasonably simulates past data distributions but also effectively predicts future trends. As Figure 9 showcases, this outcome proves the accuracy and stability of our model in capturing data patterns and verifies its generalization ability.

4.5.3. Ablation Study

To validate the impact of token masking, bidirectional encoding, clustering feature weights, and the TextCNN module on model classification performance, we constructed four model variants for ablation studies: w/o-BPE (model without bidirectional encoding), w/o-M (model without token masking), w/o-Cluster (model without cluster bias), and w/o-BPE&Mask (model lacking both bidirectional encoding and token masking modules).

As shown in

Table 8. Results of ablation studies.

Model	NVD			CCF OSC2024
	BS	ES	IS	BS	ES	IS
w/o-BPE	2.21	0.66	9.28	5.47	1.69	23.2
w/o-M	2.36	0.70	9.72	5.98	2.53	25.9
w/o-Cluster	2.19	0.63	9.13	5.54	1.17	22.0
w/o-BPE&M	2.49	0.76	10.35	6.35	2.67	28.3
LSNet (Ours)	2.15	0.61	8.97	5.59	0.93	21.4

, the primary performance improvement stems from the combined effect of bidirectional encoding and token-masking modules. This integrated approach enables the multi-scale fusion of word-order positioning and vulnerability features, achieving highly precise vulnerability severity classification through collaborative decision-making mechanisms.

5. Conclusions

In this paper, we present LSNet, an adaptive and efficient model that addresses suboptimal and inefficient prediction in vulnerability severity assessment. To uncover latent space features from vulnerability description texts, we design clustering algorithms and explicitly incorporate textual structural information into the self-attention mechanism of the Transformer-TextCNN architecture through bidirectional information encoding. Text features are comprehensively extracted by decomposing vector strings into labels via masked language modeling, which implicitly embeds them into the model. Extensive experiments demonstrate that LSNet effectively addresses traditional methods’ limitations: inaccurate assessments resulting from rigid vulnerability evaluation rules and inefficiencies caused by manual severity judgments.

Our method successfully extracts and integrates latent textual features to enhance assessment accuracy. However, this paper reveals certain limitations in the temporal modeling of vulnerability evolutionary characteristics. In this paper, we report considerable limitations in capturing and representing temporal dynamics inherent to the evolution of software vulnerabilities. Specifically, we encountered several challenges in modeling evolutionary patterns of vulnerability over time: On the one hand, the scarcity of consistently documented historical data across different software versions and platforms impedes the construction of continuous temporal sequences. On the other hand, the heterogeneous and non-uniform nature of vulnerability disclosures leads to fragmented and irregular time-series characteristics, complicating the learning of long-range dependencies. We utilize sophisticated data processing techniques and distinctive predictive approaches to decompose the comprehensive task of vulnerability severity assessment into manageable sub-problems, address them sequentially, and, ultimately, yield the final outcomes.

In future work, we will explore incorporating temporal analysis modules and integrating vulnerability lifecycle data to further optimize the timeliness and adaptability of severity assessments. We plan to extend the current framework by incorporating temporal analysis modules and integrating comprehensive vulnerability lifecycle data. These enhancements will allow the model to dynamically capture time-sensitive patterns and contextual shifts in threat landscapes.