An Intelligent Playbook Recommendation Algorithm Based on Dynamic Interest Modeling for SOAR

Abstract

With the growing demand for refined security operations, Security Orchestration, Automation, and Response (SOAR) technologies have undergone rapid advancement. By leveraging intelligent orchestration capabilities in conjunction with core playbooks, SOAR facilitates both automated and semi-automated responses to security incidents. Nevertheless, the continuous evolution of network-attack techniques and the explosive growth of security alerts have rendered traditional static rule-based playbook matching and recommendation approaches increasingly inadequate in addressing the high frequency of alerts and the emergence of novel attack patterns. In this study, we propose an intelligent playbook recommendation algorithm for SOAR, developed under the paradigm of dynamic interest modeling. Specifically, the algorithm integrates a Transformer encoder, which captures long-term dynamic characteristics of alert signals in real time, with an LSTM network designed to extract short-term behavioral patterns. This hybrid architecture not only enables accurate playbook recommendations in high-volume alert scenarios, but also supports the reconstruction and optimization of playbooks, thereby offering valuable guidance for the mitigation of emerging threats. Experimental evaluations demonstrate that the proposed dynamic interest modeling-based algorithm exhibits high feasibility. It achieves improved performance in terms of both recommendation accuracy and efficiency, thus providing a robust technical foundation for enhancing the effectiveness of network security incident response and offering practical support for real-world security operations.

1. Introduction

The rapid evolution of information technologies, coupled with increasingly sophisticated and diverse network threats, has posed significant challenges to global organizations [1]. Traditional defense mechanisms often fall short in addressing emerging security alerts, as large-scale and complex attacks can no longer be effectively mitigated through manual judgment and response. Such limitations result in delayed incident handling, weakened protection, and elevated security risks [2].

Security Orchestration, Automation, and Response (SOAR) [3] has emerged as a promising paradigm to address these challenges. As shown in Figure 1, this is a security automation response architecture based on the SOAR concept. It covers the entire process from event collection, strategy formulation, action execution, to process orchestration, capability management, and resource support, forming a closed-loop security operation system. By integrating automation and orchestration, SOAR platforms accelerate incident response, minimize human intervention, and reduce error rates. They are particularly effective in handling high-volume and high-frequency alerts through automated workflows and predefined rules. Moreover, SOAR enables cross-team collaboration by consolidating data from multiple security tools into a unified view, thereby supporting timely and informed decision-making in complex environments. In addition to enhancing operational efficiency, SOAR reduces reliance on human analysts, optimizes resource allocation, and ultimately strengthens organizational resilience against evolving network threats [4].

Playbooks represent the core component and pivotal element of SOAR technology, functioning as highly digitized security operation processes that integrate personnel, tools, devices, and workflows. In addressing increasingly complex network security incidents [5], the execution and selection of playbooks become critical for safeguarding organizational information assets. However, with the continuous evolution of attack techniques and the surge in security incidents, traditional approaches that rely on static playbooks and manual selection are no longer adequate. A key challenge lies in identifying the most appropriate playbook according to the prevailing security posture and attack patterns. In this context, the value of recommendation systems [6] has become increasingly evident. By learning from and analyzing historical data, such systems can automatically recognize attack patterns and, based on specific alerts, intelligently recommend the most suitable playbooks [7].

Through intelligent recommendation systems, security analysts can make more accurate decisions, thereby enhancing overall network security defense. Deep learning provides strong support for such systems, having already been widely applied in industrial recommendation systems for domains such as product and movie recommendations [8]. By constructing complex neural network architectures, deep learning can better capture users’ diverse interests and behavioral patterns, significantly improving recommendation accuracy and personalization [9,10]. Leveraging the multi-level feature extraction capability of deep neural networks, these models can uncover intricate user needs and behaviors from large-scale data, thereby overcoming the limitations of traditional approaches.

In the context of network security recommendation systems, deep learning algorithms can analyze historical attack incidents to identify “points of interest” or recurring “patterns,” enabling the system to better understand which security measures are most effective in specific environments and to generate more precise recommendations. The application of recommendation systems within SOAR not only introduces a more efficient and intelligent approach to security operations but also opens a new technological pathway for building adaptive and fine-grained defense mechanisms. As network-attack techniques continue to advance, intelligent recommendation systems are expected to become a critical instrument in network security operations, empowering organizations to confront increasingly complex challenges in the digital era.

In this paper, we propose a network security playbook recommendation approach based on Dynamic Interest Modeling (DIM). By analyzing and modeling multi-dimensional data from security incidents, the method leverages dynamic interest modeling to capture variations in alert-to-playbook relevance, thereby enabling more accurate playbook recommendations. As illustrated in Figure 2, the network security playbook recommendation method proposed in this paper ultimately models playbooks based on action nodes. It recommends suitable action nodes and realizes playbook recommendation by comparing the final similarity. The remainder of this paper is organized as follows. Section 2 reviews related work. Section 3 presents the overall framework. Section 4 discusses the characteristics and preprocessing of alert–playbook data. Section 5 introduces the recommendation model architecture and training procedure. Section 6 evaluates the model performance based on experimental results. Additionally, Section 6 concludes the study.

2. Related Work

Existing approaches to playbook recommendation can be broadly categorized into traditional recommendation algorithms and deep learning-based algorithms. Traditional methods mainly include item-based recommendation, collaborative filtering, and hybrid recommendation techniques. Deep learning-based approaches primarily involve intelligent recommendation models constructed with multilayer perceptrons, autoencoders, convolutional neural networks, and recurrent neural networks.

As shown in Figure 3, Pazzani [11] proposed an item-based collaborative filtering algorithm. In the field of network security, this approach has been applied to recommend historical incidents or known playbooks that are similar to the current event, based on available threat intelligence, users’ past security event data, and system logs.

As shown in Figure 4, Herlocker [12] and Sarwar [13] proposed a user-based collaborative filtering algorithm. By analyzing the historical response behaviors of security teams, this approach identifies users who have been attacked in ways similar to the current alert and leverages existing security data to assist in handling the ongoing incident.

Hybrid recommendation systems integrate content-based and collaborative filtering approaches to provide more effective security incident analysis and incident response suggestions, thereby enhancing overall network security defense capabilities. Common hybrid frameworks include WideDeep [14], DeepFM [15], Neural Factorization Machine (NFM) [16], and Deep Cross Network (DCN) [17]. While traditional recommendation algorithms can improve recommendation accuracy to some extent, they still face challenges such as cold-start issues, data sparsity, and over-fitting.

With the advancement of deep learning and big data processing technologies, intelligent recommendation systems have continued to evolve. Mao [18] proposed an intelligent recommendation algorithm, FinalMLP, based on multilayer perceptrons, which employs a dual-MLP framework and nonlinear modeling to capture deeper playbook features. Sedhain [19] introduced AutoRec, an autoencoder-based approach that learns low-dimensional representations of user–item interaction data, thereby capturing latent interest patterns and improving personalization accuracy. Wang [20] proposed DeepCoNN, a convolutional neural network-based recommendation algorithm that analyzes textual and visual cues to better understand user preferences and improve recommendation accuracy. As shown in Figure 5, Sun [21] developed an intelligent recommendation algorithm based on recurrent neural networks, which predicts user–item ratings by modeling long-term dependencies, thereby generating more accurate recommendation rankings.

Therefore, in the context of intelligent playbook recommendation for SOAR, it is essential to leverage deep learning as the core enabling technology. By utilizing its capabilities in feature learning and nonlinear modeling [22], deep learning can facilitate intelligent recommendation of security playbooks and provide dynamically adaptive incident response strategies for security teams in evolving network security environments.

3. Research Methods

3.1. Research Framework

As shown in Figure 6, this study proposes a security playbook recommendation framework based on DIM:

The proposed framework primarily takes as input two types of data: historical behavioral data of alerts and the current interaction data. Here, the interaction data refer to the attributes of the current alert and the candidate playbooks. Specifically, the alert’s historical behavioral data, such as the sequence of previously selected playbooks and the attribute sequence of past alerts, together with the feature data of the interaction items, including playbook ID, playbook attributes, alert ID, and alert attributes, are first mapped into a high-dimensional space through an embedding layer. This process transforms the raw data into dense vector representations, enabling effective processing in the subsequent network layers.

Within the framework, the Activation Unit is employed to assign weights to the historical behaviors of alerts, thereby capturing the relevance of different playbooks to the current alert. Since alert interests are inherently dynamic, the model adjusts the weights of historical playbooks according to the candidate playbook under consideration. After reweighting the historical behavior vectors, the framework aggregates them to generate a dynamic interest vector, which represents the current alert’s preference over playbook categories and attributes. This weighted dynamic interest vector is then passed through subsequent neural network layers to compute the recommendation probability for the candidate playbook, which is ultimately used to rank recommendation results.

The proposed recommendation network framework is built upon the base model paradigm of DIN [23], while incorporating the concept of dynamic interest modeling within the attention module.

3.2. Data Preprocessing and Analysis

To implement security playbook recommendation based on DIM, it is first necessary to preprocess and transform the alert behavioral data in order to construct a data set suitable for training under the DIN framework. The core objective of data preprocessing is to convert historical alert behaviors into a model-compatible format, ensuring that each alert’s behavior effectively reflects both its short-term and long-term interest dynamics.

Data Structure

In network security operations, alerts and playbooks constitute two closely interconnected key elements [24], and their relationship forms the core framework of automated security response. Alerts provide the triggering conditions for playbooks, which, in turn, execute automated responses based on the alert data. Alert data serve as the primary input in network security incident response, and the quality of their preprocessing directly affects the efficiency and accuracy of subsequent playbook recommendations.

(1) Alarm data:

As shown in

Table 1. List of alarm properties.

ID	Alarm Properties	Examples	Description
1	Alert ID	d2387y8o2hu696c	The alert ID is a unique identifier for each alert event. During the incident response process, the alert ID is used to track and manage the lifecycle of the alert, with each alert ID corresponding to a specific security event.
2	Alert Type	Cloud Firewall Attack	The alert type describes the nature of the event, such as virus infection, intrusion detection, or DDoS attack.
3	Alert Severity	High	The alert level is usually categorized as low, medium, or high, indicating the severity of the event.
4	Source IP	192.168.1.1	The source IP address of the attack or anomalous behavior.
5	Target IP	192.168.2.5	The target IP address refers to the attacked host or device, usually the victim.
6	Timestamp	2024-03-01T18:05:35.073+08:00	The event time of the alert, which helps analyze the timing of attacks, identify patterns, and detect attack windows.
7	Attack Signature	SQL Injection attack matching	Attack characteristics are the matched information between the alert and known attack patterns, typically generated by systems such as firewalls, IDS, or IPS.
8	Attack Chain Phase	Intrusion	According to the attack lifecycle model, an alert may correspond to a specific attack phase, such as reconnaissance, intrusion, or propagation.

, alarm data usually consists of multiple attributes, which not only contribute to the classification and response of alarms, but also provide rich contextual information for automated systems.

(2) Playbook data

The playbook helps security teams respond to incidents quickly and effectively through predefined steps and automated actions. As shown in

Table 2. List of playbook properties.

Id	Playbook Properties	Examples	Description
1	Playbook Name	Attack link analysis alarm notification	The unique name of the play, used for identification and reference.
2	Playbook ID	c3534638-wh9u-e5fa740bdb89	An ID that uniquely identifies the playbook and is used to associate alarms with tasks and response actions in the playbook.
3	Trigger Conditions	One violation of access was detected	A scenario triggers conditions, usually based on specific events or alerts.
4	Response Type	Notification	The response to an incident can be defensive, restorative, notification, and so on.
5	Priority	High	The priority of the playbook, indicating the urgency or importance of the processing.
6	Automated Actions	Alarm email notification	Actions that are performed automatically, often through API integration to interact with other systems.
7	Integrations Support	Integration with firewalls and intrusion prevention systems	Integration of playbook with other security tools, platforms, and services.
8	Status & Error Handling	On the fly	Handle errors or exceptions during playbook execution to ensure that the playbook is executed correctly.

, each playbook has a set of key attributes that define its trigger conditions, execution steps, priority, and more.

In handling network security alerts, traditional manual approaches perform poorly compared with SOAR playbook-based methods in terms of key indicators such as time consumption, response timeliness, and the probability of business misblocking. Specifically, the average handling time for security incidents often exceeds eight hours, the overall timely response rate remains below 50%, and the misblocking probability is high, with whitelist and blacklist management proving inefficient. As shown in Figure 7, take the publicly available step-by-step automatic response scheme as an example:

Upon receiving attack source information, the system first queries the IP whitelist. If the source address exists in the whitelist, only a notification is sent to the DingTalk duty group without further action. For addresses not included in the whitelist, the system retrieves their geolocation information and applies differentiated blocking strategies accordingly. Specifically, foreign addresses are subjected to permanent blocking, while domestic addresses are blocked in a stepwise manner based on the frequency of alerts and blacklist matching: one hit within 24 h triggers a 1 h block; two hits within 6 h trigger a 4 h block; four hits within 24 h trigger a permanent block. Any match with the blacklist, which is extracted from threat intelligence, also results in a permanent block. Finally, an automated unblocking process is executed, whereby the system lifts the block once the corresponding duration expires, using a delayed rule mechanism. By adopting this automated playbook, the response time is significantly reduced, the timeliness of responses is improved, the false-blocking rate approaches zero, and both whitelist and threat intelligence are matched in real time with high accuracy.

3.3. Alarm Behavior Sequence Construction

A complete set of alarm data for extracting valid information is shown in

Table 3. Sample alarm behavior data.

Timestamp	Alert Type	Severity	Source IP	Target IP	Playbook
2025-02-17 08:01:00	IDS	High	192.168.1.1	192.168.2.5	Intrusion detection response playbook
2025-02-17 08:03:15		High	192.168.1.1	192.168.2.7	Malware Detection and Isolation playbook
2025-02-17 08:05:30		Low	192.168.1.1	192.168.2.5	Port Scan detection and protection playbook
2025-02-17 08:07:45		High	192.168.1.1	192.168.2.6	Intrusion detection response playbook
2025-02-17 08:09:00		Low	192.168.1.1	192.168.2.5	Port Scan detection and protection playbook
2025-02-17 08:12:30		High	192.168.1.1	192.168.2.5	Malware Detection and Isolation playbook

. Alarm data first undergoes cleaning and completion to remove redundant, irrelevant, or duplicate records. For missing critical information, such as alarm type or timestamp, missing values are either imputed or the corresponding records are discarded to ensure data consistency. After this step, alarm behaviors are sorted chronologically and organized into time-series data. The sequence of alarms reflects the temporal progression of potential attacks in the network and their evolving paths, which directly influence subsequent analysis and response decisions.

To better capture the dynamic nature of alarm behaviors and represent their temporal variations, this study adopts a sliding window approach. The long-term alarm sequence is segmented into multiple shorter time intervals, each corresponding to a localized set of alarm behaviors and interest patterns. This method effectively extracts evolving behavioral trends from local temporal contexts and provides them as model inputs.

The sliding window procedure consists of the following steps:

• Determining the window size: First, the window size needs to be determined according to the practical requirements of network security operations. The choice of window size depends on the frequency of security events as well as business needs. A smaller window can capture more fine-grained dynamic variations, while a larger window is capable of reflecting long-term behavioral patterns. In this study, we use alarm data with a five-minute window to represent the short-term interests of the current alarm and a one-week window to represent its long-term interests.
• Window sliding with segmentation: On the basis of a fixed window size, the sliding window divides the alarm behavior sequence into multiple time intervals. For example, within each five-minute window, the system collects all historical response data associated with the alarm during that period and generates a feature vector based on this data to represent the alarm behavior within the corresponding time segment.

Through the sliding window mechanism, variations in alarm behaviors along the temporal dimension can be observed, allowing the capture of both short-term and long-term interest patterns. Each sliding window output is treated as an independent behavioral sequence, which contains the features and patterns of alarm data occurring within that window. These behavioral sequences serve as input to the DIN framework, and through further model training, they enable the prediction and recommendation of appropriate security playbooks.

Taking a window length of 5 min with a 1 min step size as an example, two processed historical alarm records are generated, as shown in

Table 4. Historical behavior sample data.

Window 1: 17 February 2025 08:00:00 to 17 February 2025 08:05:00
Source IP	[192.168.1.1, 192.168.1.1, 192.168.1.1]
Target IP	[192.168.2.5, 192.168.2.7, 192.168.2.5]
Playbook	[Intrusion detection response playbook, malware detection and isolation playbook, port scan detection and protection playbook]
Window 2: 17 February 2025 08:02:00 to 17 February 2025 08:07:00
Source IP	[192.168.1.1, 192.168.1.1, None]
Target IP	[192.168.2.5, 192.168.2.7, None]
Playbook	[Malware detection and isolation playbook, port scan detection and protection playbook, None]

. Through the sliding window approach, the alarm behavior data are divided into two time windows, and relevant features are extracted within each window. These features constitute the input data set for model training, where each time window represents localized security activities within the network. Each feature vector includes information such as alarm type, alarm level, source/target IP, and previously applied playbooks, enabling the model to capture long-term trends in alarm behavior. Meanwhile, the corresponding playbook information reflects the short-term variations in alarm behavior.

3.4. Data Encoding Method

The features of alarm behavior data, behavioral sequences, and playbook characteristics are typically unstructured or categorical in nature. To ensure these data can be effectively utilized by machine learning models, they must be appropriately encoded. Alarm data contain a large number of categorical features, such as alarm type, alarm level, source IP, and destination IP, all of which are non-numeric. In order for the model to interpret these features, they need to be transformed into numerical representations.

In this study, One-Hot Encoding [25] is adopted. One-Hot Encoding transforms categorical features into binary vectors by creating a new feature column for each possible category. If a category is present in a given sample, the corresponding column is assigned a value of 1; otherwise, it is set to 0. This encoding method is generally suitable for categorical features without intrinsic ordinal relationships. For instance, considering the categorical feature of alarm level with values low, medium, and high, the One-Hot Encoding representation [High, Medium, Low] can be expressed as [1, 0, 0], [0, 1, 0], and [0, 0, 1], respectively.

4. Model Training for Playbook Recommendation

4.1. Core Mechanism of Long- and Short-Term Interest Modeling

In the domain of recommendation systems, dynamic interest modeling [26] constitutes a pivotal mechanism for characterizing the temporal dynamics of user preferences. Within the context of security playbook recommendations, the decision-making tendencies of security operation analysts can be analogized to alert-driven interests. To this end, the present study differentiates such interests into long-term and short-term components, as depicted in the Figure 8. The model traces the lifecycle of an alert-from its initial occurrence to its most recent activations-while incorporating a sequence of temporally annotated playbook response records, thereby providing a comprehensive representation of evolving alert-related preferences.

Taking the lifecycle of two representative alerts as illustrative examples, the historical records reveal distinct playbook responses occurring at different timestamps. The duration of the time span reflects the heterogeneity in alert-specific interest patterns toward playbooks.

In this study, the historical playbook response records requiring long-term monitoring are defined as the long-term behavioral profile of an alert, whereas the response records triggered frequently within a short time span are regarded as the short-term behavioral profile. Both profiles are explicitly modeled and subsequently incorporated as the long-term and short-term interest features of the alert. To capture the temporal dynamics of such alert interests, a sliding-window mechanism is employed to construct behavior sequences in real time or near real time, rather than relying on static profiling.

(1) Long-term interest in modeling design

Long-Term Interest Modeling aims to uncover the stable strategic preferences developed by security teams during continuous operations. Such preferences are often manifested as habitual responses to specific attack patterns. For instance, some teams consistently employ customized waf-rule playbooks for web-based attack alerts, while strictly adhering to zero-trust isolation policies when addressing lateral movement behaviors within the internal network. To cope with the real-time and dynamic nature of such scenarios, this study adopts a Transformer-based architecture, whose technical advantages can be summarized as follows.

The self-attention mechanism of the Transformer overcomes the locality constraints inherent in traditional sequence models. When processing playbook sequences, the model does not only focus on adjacent playbook events but also establishes global dependencies across temporal steps. For example, when the system encounters a current alert requiring an “abnormal database access response playbook,” the model can automatically associate it with a “SQL injection attempt response playbook” triggered a week earlier, thereby recommending the previously validated effective playbook. This capability of capturing long-range dependencies is achieved through multi-head attention, where historical playbooks and candidate playbooks are encoded into query (Q), key (K), and value (V) triplets, and their similarities are dynamically weighted. Compared with the serial processing of RNNs, the parallel computation of Transformers significantly enhances efficiency.

In the context of security operations, the Transformer can analyze the usage frequency of different playbook types-such as whether a certain alert is consistently handled by a specific playbook-and track the evolution of handling habits. This ensures that recommended playbooks remain consistent with long-term strategies, while preserving adaptability of security policies. Consequently, the system can optimize recommendation decisions based on historical behaviors, particularly when alert types exhibit relative stability or repeated occurrences.

(2) Short-term interest in modeling design

In network security operations, the primary objective of short-term interest modeling is to capture, in real time, the dynamic behavioral preferences of security analysts during alert handling. Such preferences are often driven by emergent attack events. For example, when a novel vulnerability exploitation attack frequently occurs within a short period, security teams may prioritize specific emergency response playbooks, such as isolating infected hosts or rapidly deploying temporary patches. To effectively capture these short-term patterns, this study employs an LSTM-based framework, whose design incorporates several key innovations.

First, the model leverages gating mechanisms to dynamically filter memory. The forget gate of LSTM automatically identifies and attenuates outdated strategic preferences. For instance, if a phishing-response playbook has not been invoked in the past six months, its weight will be gradually reduced. Conversely, the input gate continuously assimilates new effective experiences, such as recently optimized playbooks for container escape detection in cloud environments. This mechanism effectively addresses the common issue in traditional methods where obsolete strategies remain erroneously influential.

Second, the LSTM encodes recent historical playbook behaviors and computes the contribution weight of each historical action with respect to the current candidate playbook. These weights are then aggregated to generate a short-term interest representation, which serves as the basis for recommending the most contextually appropriate playbook. Through this design, the system can rapidly adapt to sudden changes in the alert landscape, ensuring both timeliness and effectiveness in automated response.

(3) The sliding window extracts long- and short-term interests

As shown in Algorithm 1, by using the sliding window method, the “long-term interest” and “short-term interest” behavior patterns are extracted from the historical alarm data, providing data support for the subsequent recommendation of cybersecurity scripts based on dynamic interest modeling.

Algorithm 1. The sliding window method is used to extract alarm behavior records

Input: Historical data of all alarms in json format

Output: A record of long and short term behavior of all alarms

1: define the long and short interest time span LONG_TERM_SPAN, SHORT_TERM_SPAN

2: for iterates over all alarms

3:     Extract the lifetime of the current alarm (including all the corresponding records of the playbook)

4:     define The current time variable current_time

5:     While: current_time during the lifetime:

6:         Long-term window extraction

7:         Short-term window extraction

8:         Sliding step size according to LONG_TERM_SPAN, SHORT_TERM_SPAN

9:     Record the current alarm result

10: return alert long-term interest history, alert short-term interest history

4.2. Intelligent Recommendation Framework Based on Dynamic Interest Modeling

The recommendation framework adopted in this study is an improved version of DIN (Deep Interest Network), in which the concept of dynamic interest modeling is introduced into the attention module. Within this framework, alerts are analogous to users in a conventional recommendation system, while playbooks correspond to candidate items. Although rule-based static matching methods can satisfy compliance requirements, they often fail to address the evolving threat characteristics of novel attacks [27]. To overcome this limitation, we propose a dynamic interest-based playbook recommendation algorithm, which incorporates dynamic interest modeling into the Activation Unit. This design enables the recommendation system to more flexibly respond to the short-term behavioral patterns of alerts, thereby enhancing the precision of behavioral feature capture and improving adaptability in network security playbook recommendation tasks.

The main components of the framework are as follows:

(1) Embedding Layer

The embedding layer primarily converts discrete features into dense vectors, allowing subsequent processing within the deep neural network. The network input consists of historical alert behaviors, playbook IDs, playbook attributes, alert IDs, and alert attributes. The current interaction serves as the query item within the model, which is matched against historical behaviors.

Single-value features, such as alert type and playbook ID, are each represented by an independent embedding vector.

$$e_x=f_{emb}\left(x\right)$$

(1)

Specifically, $\mathrm{x}$ denotes the input discrete feature, and $f_{emb}\left(x\right)$ represents the embedding layer, which maps discrete features into a low-dimensional vector space to obtain the embedding vector $e_x$.

For multi-valued features, such as the historical sequence of playbooks clicked by alerts, multiple embedding vectors are aggregated using either summation or average pooling. The embedded features are then projected into a high-dimensional dense space, enabling their subsequent processing within the neural network. For the historical playbook click sequence $H=\left\{h_1,h_2,...,h_m\right\}$, where each $h_i$ denotes a previously clicked playbook, the embedding transformation process is expressed as follows:

$$E_H=\left\{e_{h_1},e_{h_2},\cdots ,e_{h_m}\right\}$$

(2)

Here, $e_{h_1}$ denotes the embedding vector of a playbook. The candidate playbook is also transformed into a vector representation through the embedding layer, where $t$ represents the playbook currently under recommendation:

$$e_t=f_{emb}\left(t\right)$$

(3)

The embedding layer is implemented based on a lookup table, which maps index values to a weight matrix of a specific dimension. By optimizing the dense vector representations of discrete features, the model captures the latent relationships among features. During training, each row of the embedding matrix is iteratively updated so as to optimize the overall task objective.

(2) Activation Unit

This module constitutes the core of the proposed algorithm, where a self-attention mechanism is employed to jointly model short-term and long-term interests for generating dynamic interest representations. The self-attention mechanism is designed to capture long-range dependencies within the input data, enabling the measurement of correlations between historical alert behaviors and the currently interacting playbook [28]. In the context of security playbook recommendation, self-attention allows the model to dynamically adjust the degree of emphasis placed on different historical playbooks with respect to the candidate playbook, thereby enhancing the precision of recommendations.

Within the DIN framework, the primary function of the self-attention mechanism is to compute the relative importance of historical behaviors with respect to the current candidate item, assigning distinct weights to different interest signals. Specifically, the mechanism assigns differentiated weights to the playbooks previously triggered by the alert, such that historical playbooks most relevant to the candidate playbook are given higher attention scores. The attention weights between each playbook in the historical sequence and the candidate playbook are first computed, and a weighted summation is then performed to generate a comprehensive interest representation vector. In this manner, the model is able to flexibly capture the evolution of alert interests under different security contexts.

$${\alpha }_i={\displaystyle \frac{exp\left(f\left(e_{h_i},e_t\right)\right)}{{\sum }_{j=1}^{m}exp\left(f\left(e_{h_j},e_t\right)\right)}}$$

(4)

${\alpha }_i$ denotes the attention weight assigned to a historical playbook with respect to the current target playbook. The function $f\left(e_{h_i},e_t\right)$ represents an adaptive interest activation function, which is employed to compute the correlation between the candidate playbook and historical behaviors. In practice, this function is commonly implemented using a multilayer perceptron (MLP):

$$f\left(e_{h_i},e_t\right)=MLP\left(e_{h_i}⨀e_t\right)$$

(5)

$⨀$ denotes the element-wise vector multiplication (Hadamard product), which is employed to compute the similarity between the candidate playbook and each historically clicked playbook.

Based on the attention weights $\alpha$, the weighted interest representation is then obtained as:

$$v=\sum _{i=1}^m{\alpha }_i{e}_{h_i}$$

(6)

$v$ denotes the final playbook interest representation, which dynamically depends on the current target playbook $e_t$. In other words, the attention focus of an alert varies according to the playbook being recommended.

Through the self-attention mechanism, the model can not only effectively eliminate redundant information within the historical playbook sequence but also identify the playbooks that exert the greatest influence on the current interaction, thereby enhancing the reliability of the recommendation results. Moreover, this mechanism improves the generalization capability of the recommendation system, enabling it to better adapt to the continuously evolving landscape of security threats.

(3) Fully Connected Layers

The prediction layer serves as the output module of the model, responsible for transforming the input interest representation into the interaction probability between an alert and the current candidate item. Its input is the alert interest representation obtained through self-attention-based weighting and fusion. Specifically, the inputs consist of the following components:

Representation of the current interaction item: This typically refers to the embedding representation of the candidate playbook associated with the alert, encompassing both the item ID and other relevant feature embeddings.

Weighted historical behavior representation: This is derived from the self-attention mechanism, capturing the relationships between the current playbook and the most relevant items in the historical playbook sequence. By leveraging self-attention, the model dynamically emphasizes the historical playbooks exerting the greatest influence on the current interaction.

$$p=\sigma \left(MLP\left(\left[v,e_t,x_{other}\right]\right)\right)$$

(7)

$p$ denotes the final click-through rate (CTR) prediction, $v$ represents the interest representation of the alert, and $e_t$ refers to the embedding vector of the candidate playbook. The function $\sigma$ corresponds to the Sigmoid activation, while $x_{other}$ denotes additional input features, such as the basic attributes of the alert.

By integrating these two sources of information, the prediction layer is able to derive a precise interest representation, which reflects the alert’s inclination toward the current candidate playbook. The fused representation is subsequently passed through a multi-layer perceptron (MLP) to further extract high-dimensional features and to generate the final prediction outcome. The framework adopts the standard binary cross-entropy loss function as its optimization objective:

$$Loss=-{\displaystyle \frac{1}{N}}\sum \left(y\log p+\left(1-y\right)\log \left(1-p\right)\right)$$

(8)

$p$ denotes the predicted output, and $y$ represents the expected ground-truth value.

Overall, the recommendation process is as shown in Algorithm 2 below. The input data consist of static alert features, playbook features, and the sequence of historical alert behaviors. After embedding, the playbook features are concatenated with the historical behavior sequence to serve as the input to the activation unit. The output of the activation unit corresponds to the attention weights assigned to each historical behavior. These weighted representations are then aggregated and concatenated with the playbook features and static alert features, which together form the input to the prediction layer. Finally, the model outputs the matching probability between the current alert and the candidate playbook.

Algorithm 2. Recommendation Framework Based on Dynamic Interest Modeling

Input: static alert features user_features, playbook features item_features, historical alert behavior sequence user_behavior_seq

Output: The probability that the alarm matches the playbook

1: define the attention unit attention_mlp

2:     Complete the playbook feature embedding e_c

3:     Complete the alarm history behavior sequence embedding e_i

4:     stitching characteristics concat_features = e_i||e_c||(e_i-e_c)||(e_i × e_c)

5: calc calculates attention scores attention_scores=attention_mlp(concat_features)

6:     Weighted aggregated alarm interest representation v_u = e_i × attention_scores

7:     combined = v_u||e_c||user_features

8:     combined is fed into the fully connected layer to predict the matching probability

9: return prediction probability

4.3. Implementation of Recommendation Algorithm Based on Dynamic Interest Modeling

The core of the framework lies in the self-attention mechanism, which aims to disentangle the intrinsic correlations between security alerts and historical incident response playbooks, thereby providing a cognitive foundation for dynamic decision-making. The current candidate playbook functions as the query signal, which performs pattern matching within the key–value knowledge base constructed from historical playbook sequences. For instance, when detecting ransomware encryption behavior, the system automatically associates historical response records containing similar attack attributes, such as file types or encryption algorithms, while attenuating the influence of outdated strategies.

The self-attention mechanism module is shown in Figure 9. In the attention activation unit, a dynamic fusion architecture is designed: the short-term interest module focuses on the real-time context of attack events, capturing dynamic features such as the propagation paths of emerging exploit tools or the phase-specific variations in adversarial tactics. In contrast, the long-term interest module emphasizes the strategic preferences of security teams, preserving stable recognition of recurrent attack patterns, such as DDoS [29] traffic signatures or phishing email detection rules. The collaboration between the two is not a simple weighted summation; rather, it is achieved through a fully connected network that dynamically adjusts based on situational awareness. During highly adversarial phases, the system automatically amplifies the decision weight of the short-term module to rapidly adapt to battlefield changes, whereas in stable periods it prioritizes the robustness provided by long-term strategies. The overall network architecture is illustrated in the following figure:

(1) Long-term interest modeling implementation

Long-term interests reflect the stable preferences formed by alerts over a relatively long time scale, and these preferences are usually closely related to the inherent attributes of alerts and the long-term attributes of historical playbooks, such as playbook types. In network security operations, continuous attention to alert attributes and playbook attributes can reveal their long-term interest patterns. To effectively capture such long-term interests, this paper thus introduces the Transformer encoder for the modeling of long-term interests.

The Transformer encoder is a deep learning model based on the self-attention mechanism, which can effectively capture long-range dependencies in input sequences. In the modeling of long-term interests, the Transformer encoder can calculate the importance of each historical item according to the current candidate playbooks and the historical click behaviors of alerts through the self-attention mechanism, and dynamically adjust the weights of historical items.

The input data mainly consists of the sequence $H=\left\{h_1,h_2,...,h_m\right\}$ of historical interactive playbooks related to alerts. Through embedding, it is converted into $E_H=\left\{e_{h_1},e_{h_2},...,e_{h_m}\right\}$. The currently recommended playbooks are also transformed into $e_t=f_{emb}\left(t\right)$. These historical playbooks reflect the behaviors of alerts under different circumstances, specifically whether a certain playbook was clicked when responding to the alert.

In the Transformer encoder, the self-attention mechanism is employed to calculate the correlation between the current candidate playbooks and historical playbooks, thereby dynamically adjusting the weights of the historical playbooks.

Firstly, the Query, Key, and Value are computed as follows:

$$Q=W_Q{e}_t,K_i=W_K{e}_{h_i},V_i=W_V{e}_{h_i}$$

(9)

Among them, $W_Q$,$W_K$,$W_V$ denotes the learnable parameter matrix. $Q$ represents the query vector of the current candidate playbook. $K_i,V_i$ stand for the key and value vectors of historical playbooks, respectively. Next, the attention score between Query and Key is calculated:

$${\alpha }_i={\displaystyle \frac{exp\left(\frac{Q·K_i}{\sqrt{d_k}}\right)}{{\sum }_{j=1}^{n}exp\left(\frac{Q·K_j}{\sqrt{d_k}}\right)}}$$

(10)

wherein, $d_k$ is the dimension of the key vector, which is used for scaling to prevent gradient explosion. ${\alpha }_i$ represents the attention weight of the i-th historical playbook, indicating the extent of its influence on the current candidate playbook.

To enhance the robustness and consistency of attention weights, a symmetry regularization term is introduced, which penalizes the asymmetry between the forward and reverse attention matrices.

Specifically, the loss function is defined as:

$$\begin{gathered} {\mathcal{L}}_{\mathrm{s}\mathrm{y}\mathrm{m}}^{\mathrm{T}}={\left|\right|Att(Q,K)-Att(K,Q)\left|\right|}^2 \\ {\mathcal{L}}_{\mathrm{t}\mathrm{o}\mathrm{t}\mathrm{a}\mathrm{l}}={\mathcal{L}}_{\mathrm{r}\mathrm{e}\mathrm{c}}+{\mathsf{\lambda }}_{\mathrm{T}}{\mathcal{L}}_{\mathrm{s}\mathrm{y}\mathrm{m}}^{\mathrm{T}} \end{gathered}$$

(11)

where ${\mathcal{L}}_{\mathrm{r}\mathrm{e}\mathrm{c}}$ denotes the original task loss and ${\mathcal{L}}_{\mathrm{s}\mathrm{y}\mathrm{m}}^{\mathrm{T}}$ is the symmetry regularization term.

This design enforces bidirectional consistency in the attention mechanism while maintaining the original attention computation structure.

Finally, the representation of long-term interests $S$ is obtained through weighted summation:

$$S={\sum }_{i=1}^n{\alpha }_i{V}_i$$

(12)

$S$ represents the long-term interest representation, which reflects the dynamic interest of the current candidate playbook in historical playbooks. The output after the Transformer encoder is a feature expression containing long-term interests, which captures the recent behavioral patterns and preferences of alerts in security incident response. This representation reflects the long-term interests corresponding to the current alert type, and the subsequent structure can dynamically adjust the interest weights according to the characteristics of the alert.

(2) Short-term interest modeling implementation

Short-term interests mainly reflect the recent playbook click dynamics of alerts, especially their performance in the process of alert response. In network security operations, the behaviors of alerts can change rapidly; therefore, it is necessary to flexibly capture and model such changes. Traditional interest modeling methods are relatively difficult to handle the rapid changes in short-term behaviors, so this paper adopts LSTM for modeling.

In the modeling of short-term interests, the input is the attribute sequence of historical playbooks:

$$A=\left\{{\alpha }_1,{\alpha }_2,...,{\alpha }_n\right\}$$

(13)

wherein, ${\alpha }_i$ represents the attribute vector of the i-th historical playbook. In this paper, an embedding layer is used to map discrete features to the vector space:

$e_{{\alpha }_i}$ represents the feature embedding of the attributes of playbook ${\alpha }_i$.

LSTM is an RNN architecture suitable for modeling sequence data, which can calculate the short-term preferences of alert historical attributes through the forget gate, input gate, and output gate.

The forget gate determines which information in the attributes of past historical playbooks should be retained and which should be forgotten:

$$f_t=\sigma \left(W_f·\left[h_{t-1},e_{{\alpha }_t}\right]+b_f\right)$$

(14)

$f_t$ is the activation value of the forget gate, ranging from (0, 1), which is used to control which historical information should be forgotten. $W_f,b_f$ are the parameters of the forget gate. $h_{t-1}$ is the hidden state at the previous time step and the previous short-term interest representation. $e_{{\alpha }_t}$ is the alert attribute vector at the current time step.

The input gate determines how the new playbook information at the current time step should be stored in the short-term memory:

$$i_t=\sigma \left(W_i·\left[h_{t-1},e_{{\alpha }_t}\right]+b_i\right)$$

(15)

$${\tilde{C}}_t=tanh\left(W_c·\left[h_{t-1},e_{{\alpha }_t}\right]+b_c\right)$$

(16)

$$C_t=f_t·C_{t-1}+i_t·{\tilde{C}}_t$$

(17)

$i_t$ is the activation value of the input gate, which determines the weight for storing new information. ${\tilde{C}}_t$ is the new memory candidate at the current time step, representing the short-term interest contribution value of the current playbook. $C_t$ is the short-term memory state, which combines the past historical interests and the current playbook information.

$$o_t=\sigma \left(W_o·\left[h_{t-1},e_{{\alpha }_t}\right]+b_o\right)$$

(18)

$$h_t=o_t·tanh\left(C_t\right)$$

(19)

$o_t$ is the activation value of the output gate, which determines how the short-term interests affect the interest representation at the current time step. $h_t$ is the hidden state at the current time step, i.e., the short-term interest representation.

The final short-term interest representation is determined by the hidden state of the LSTM at the last time step:

$$L=h_n$$

(20)

In short-term interest modeling, LSTM can identify short-term preferences for playbook types, categories, and security levels by processing time-series data of historical playbook information. Through LSTM, information such as playbook types and categories can be processed and transformed into the short-term interest representation of alerts, which reflects the interest trends of alerts over a period of time.

(3) Dynamic synergy of long- and short-term interests

In the framework of this paper, the collaboration between long-term and short-term interests is mainly achieved by concatenating the long-term and short-term interest vectors and inputting them into the MLP layer to calculate dynamic weights. Based on the learning of alert information by the MLP layer, the network can adaptively adjust interest weights in different scenarios: in regular operation and maintenance phases such as daily traffic monitoring, the recommendation strategy is dominated by long-term interests, focusing on ensuring the stability and compliance of security strategies; in emergency response scenarios such as sudden high-risk attack incidents, the system dynamically enhances the decision-making weight of short-term interests and prioritizes the recently verified effective disposal modes. This dynamic collaboration mechanism balances historical experience and real-time feedback, which not only avoids the rigidity of long-term strategies but also prevents the short-sightedness in decision-making caused by short-term behaviors, thus being more flexible in complex attack-defense confrontations.

The process of the activation unit is shown in the Algorithm 3. The short-term interest behaviors of alerts and playbook features are concatenated to form short-term inputs, which are then fed into the LSTM encoder. In this paper, the concatenated result of the last two hidden states of the LSTM is taken as the extracted short-term interest features. The long-term interest behaviors of alerts and playbook features are concatenated to form long-term inputs, which are input into the Transformer encoder to extract global long-term interest features. Finally, the short-term features and long-term features are concatenated, and the weighted weights of the historical behavior sequence are generated through the fully connected layer, reflecting the importance of behaviors at different time steps to the current alert.

Algorithm 3. Activates the unit module process

Input: alert long-term interest history, alert short-term interest history, playbook feature item_features

Output: The weighted weight of the alarm historical behavior sequence

1: define the short-term interest modeling layer LSTM_encoder

2: define the long-term interest modeling layer transformer_encoder

3:       Concatenate short-term interest behavior and playbook features as the input of LSTM_encoder combined_input_short

4:       Concatenate long-term interest behavior and playbook features as input to transformer_encoder combined_input_long

5: calc calculate long-term featureslong_term_interest=transformer_encoder(combined_input_long)

6:       rnn_out,(h_n,c_n)=LSTM_encoder(combined_input_short)

7:       Take the last hidden state as the short-term feature short_term_interest=h_n[−2]||h_n[−1]

8:       combined_interest=short_term_interest||long_term_interest

9:       The long-term and short-term features are fed into the fully connected layer to obtain the weighted weight Weight_history of the historical behavior

10: return Weight_history

5. Experimental Results and Analysis

5.1. Evaluation Index

(1) Precision: Refers to the proportion of alert playbooks recommended by the system based on dynamic interest modeling that are judged to be applicable to the current alert and are actually applicable. It reflects the accuracy of “effective playbooks” in the dynamic recommendation results.

(2) Recall: Refers to the proportion of all valid playbooks related to the current alert that are successfully recommended by the system through dynamic interest modeling. In other words, it represents how much key content the dynamic model can cover among all applicable playbooks corresponding to the alert.

(3) F1 Score: The harmonic mean of Precision and Recall, which comprehensively measures the performance of dynamic interest modeling in two aspects: first, the proportion of effective playbooks in the recommendation results; second, the ability to cover all applicable playbooks. It balances the needs of “accurate recommendation” and “comprehensive coverage”.

(4) AUC: Evaluates the dynamic interest model’s ability to distinguish between “applicable playbooks” and “non-applicable playbooks” in sequential decision-making through the area under the ROC curve (the relationship between True Positive Rate (TPR) and False Positive Rate (FPR) under different thresholds). As a core indicator for evaluating the ranking quality of recommendation systems, it particularly reflects the dynamic model’s ability to prioritize high-quality playbooks during the interest evolution process.

(5) Average Precision: AP is an indicator for measuring the accuracy of a recommendation system, reflecting whether the system can prioritize recommending playbooks applicable to the alert. For each query, AP calculates the precision at different recall rates and then takes the average of these precision values.

(6) Hit Ratio: The hit ratio measures whether the recommendation system successfully recommends at least one playbook applicable to the alert. It focuses on whether a hit occurs rather than the specific ranking of recommendations or precision.

(7) Normalized Discounted Cumulative Gain (NDCG): NDCG [30] is an indicator that considers the order and relevance level of recommended playbooks. It can measure whether the system ranks the most relevant playbooks at the top, as it assigns different weights to the ranking positions in the recommendation list.

(8) Mean Average Precision (MAP): MAP [31] is the average of AP values across all queries. It combines the precision and ranking of the recommendation system and is applicable to the evaluation of various recommendation tasks. AP(i) is the AP of the i-th query.

(9) Mean Reciprocal Rank (MRR): MRR is an indicator for evaluating the ranking task of a recommendation system. It mainly focuses on the rank of the first relevant playbook in the predicted list.

5.2. Data Set

5.2.1. Movielens Data Set

The MovieLens data set [32] is a classic movie rating data set that includes users’ ratings and viewing records of movies, as well as information about the movies themselves. Built based on users’ movie rating behaviors, the MovieLens data set is formed into a database containing fields such as user ID, movie ID, rating, and timestamp by collecting and organizing a large amount of users’ rating data. With a wide range of data sources, the MovieLens data set covers users from multiple countries and regions, ensuring the diversity and representativeness of the data set.

5.2.2. The Alert-Playbook Data Set

The Alert-Playbook Dataset is a specialized dataset developed by real cloud service providers based on actual business scenarios in real-world network environments. It contains 100,000 valid samples, with the following class distribution: among all alerts, host status alerts account for 45%, Web application defense alerts account for 30%, attack chain alerts account for 20%, and other types of alerts account for 5%. In terms of alert severity, high-severity alerts account for 20%, medium-severity alerts account for 50%, and low-severity alerts account for 30%.

This dataset is used to simulate and analyze abnormal events (alerts) and response strategies (playbooks) in cloud computing environments. By systematically integrating massive operational data and security logs, it provides a solid data foundation for risk management and intelligent decision-making in cloud environments. The Alert-Playbook Dataset mainly includes core elements such as alert type, alert title, alert severity, attack type, and playbook name, covering various disposal playbooks, including host status alerts, Web application defense alerts, and attack chain alerts.

5.3. Feasibility and Results

Feasibility Analysis

To verify whether network security operation playbook data can be effectively used in an intelligent recommendation framework, this paper compares the recommendation performance of six representative recommendation models—namely DeepFM, WideDeep, DCN, NFM, AFM (Attentional Factorization Machines), and AutoInt—on both the Alert-Playbook data set and the traditional recommendation data set MovieLens, as shown in Figure 10. This is done to validate the adaptability and effectiveness of the recommendation system in security operation scenarios.

To verify the convergence of recommendation models on the Alert-Playbook data set, this paper compares the loss curves of each recommendation model on the Alert-Playbook data set and the traditional MovieLens data set. On the Alert-Playbook data set, the loss curves of all recommendation models show an overall stable downward trend, indicating that the models can effectively learn the matching relationship between alerts and playbooks and gradually converge. Furthermore, compared with the traditional MovieLens data set, the loss curves of the Alert-Playbook data set exhibit a faster convergence rate in some models, which may be related to the feature structure and high relevance of security alert data.

To further explore the performance differences in various recommendation algorithms in network security operation scenarios, this paper conducts comparative experiments on each recommendation model using multiple evaluation metrics. In security scenarios such as security playbook recommendation—characterized by timeliness, sequentiality, and time-sensitivity—apart from the commonly used metrics including Precision, Recall, F1 Score, AUC, and AP, this paper also introduces four additional metrics: Hit Ratio, NDCG, MAP, and MRR. These metrics enable a more comprehensive evaluation of the models across four dimensions: accuracy, recall capability, ranking quality, and rapid response capability.

The results show that there are significant differences in the adaptability of different recommendation algorithms in network security operation scenarios: NFM performs excellently in multiple evaluation metrics such as Precision, F1 Score, and AUC, demonstrating the advantage of strong comprehensive performance. This is mainly attributed to the fact that NFM effectively combines the second-order feature interaction of FM (Factorization Machines) and the higher-order feature learning of neural networks. It is suitable for accurately modeling alert interests, and its recommendation effect is more precise and stable. Therefore, it is particularly applicable to security playbook recommendation tasks with high requirements for accuracy.

AFM stands out in ranking-related metrics such as NDCG, MAP, and MRR, which indicates that AFM is better at mining important feature interaction relationships through the attention mechanism and improving the ranking quality of recommendation results. It is especially suitable for network security recommendation tasks where optimizing the ranking effect of playbooks is desired.

Although DIN ranks at a medium-to-high level in overall performance, it shows obvious advantages in capturing dynamic interests and adjusting recommendations in real time. The DIN model dynamically adjusts the weights of historical behaviors through the attention mechanism, enabling it to more accurately capture the rapidly changing interest preferences of security alerts. However, its performance stability is insufficient when dealing with more complex interest modeling tasks. Therefore, DIN is more suitable for scenarios with high requirements for dynamic response capabilities, such as playbook recommendations for emergency security events like 0-day attacks.

To better compare the performance of different recommendation models, this paper plots a radar chart as shown in the figure. It can be seen from the radar chart that NFM and AFM perform prominently in multiple key metrics. Particularly in indicators measuring recommendation quality, such as Precision, Recall, F1 Score, AUC, and AP, they are significantly superior to other models. In particular, NFM performs best in AUC and AP, indicating that it has a stronger ability to distinguish between positive and negative samples, and its recommendation results are more relevant.

DIN maintains good performance in metrics like Precision, F1 Score, and AUC, which is better than DeepFM and WideDeep. However, its performance in MAP and MRR is average, suggesting that there is still room for optimization in terms of recommendation ranking. AutoInt shows a relatively balanced overall performance, with moderate scores in Precision, Recall, and AUC, but it achieves a high score in NDCG, indicating that the ranking quality of its recommendations is relatively good.

As shown in Figure 11, in ranking-related metrics, including NDCG, MAP, and MRR, AFM obtains the highest scores, demonstrating that this model not only recommends more relevant playbooks but also performs better in terms of recommendation order, making the most relevant playbooks more likely to be prioritized.

On the whole, the experimental results verify the effectiveness and applicability of network security operation playbook data under the recommendation framework, and different recommendation algorithms show differentiated advantages in security operation scenarios. Among them, AFM and NFM perform excellently in multiple metrics, especially achieving high scores in Precision, Recall, AUC, F1 Score, AP, and NDCG, indicating that they can effectively balance precision and recall and perform well in recommendation ranking. DIN, on the other hand, performs well in MAP and MRR, making it more suitable for scenarios that require flexible responses to sudden security incidents. However, for historical data of time-series alert behaviors, NFM itself does not inherently have the ability to process time-series data and needs to introduce time information through feature engineering. Through such means, NFM can handle some historical interaction data, but it cannot automatically capture long-term trends or dynamic changes in time series because it lacks the capability of time-series modeling. Therefore, to better handle time-series data and dynamic interest changes, it is necessary to introduce recommendation models capable of handling dynamic changes, such as DIN, or models specifically designed for time-series processing, such as LSTM, GRU, and Transformer.

Since the core goal of this study is to improve the adaptability of security playbook recommendations in sudden security incidents, the DIN network was subsequently selected as the basic framework. On this basis, Transformer and LSTM were further introduced for improvement—with the former enhancing the capture of long-term interest associations and the latter strengthening the modeling of sequential interest changes—to boost the model’s ability to model dynamic interests. For model training, key hyperparameters were set as follows: a learning rate of 1e-4, the Adam optimizer, a batch size of 64, and a total of 100 training epochs, which ensured stable convergence while laying the foundation for improving recommendation precision and timeliness.

This paper compares and analyzes the loss curves and the performance of various evaluation metrics during training and validation of the original DIN model and the improved model after introducing DIM [33] (DIN+DIM), as shown in the following figures.

By observing the loss curves, it is found that the loss of the DIN model on the validation set decreases slightly, and the validation loss is higher than the training loss. This phenomenon is closely related to the fixed-weight loss function used in the original DIN: the function treats all misclassified samples equally, failing to emphasize the samples that reflect critical security scenarios. As a result, the model over-fits to general samples, leading to insufficient generalization performance.

In contrast, for the DIN+DIM model with dynamic interest modeling, we first optimized the loss function by introducing dynamic weight adjustment mechanisms: the weight of each sample in the loss calculation is dynamically adjusted based on two factors—the urgency of the alert and the degree of matching between the model’s predicted interest and the actual playbook demand. This optimized loss function enables the model to focus more on key samples that affect generalization, thus making its validation loss decrease significantly and approach the training loss curve. This directly proves that the optimization of the loss function, combined with dynamic interest modeling, effectively alleviates over-fitting and improves the model’s generalization ability.

To further prevent over-fitting, this paper adds an early stopping trigger mechanism based on the optimized loss function: if the validation loss (calculated by the dynamic weight loss function) increases by more than 5% (threshold set based on preliminary experiments) for 3 consecutive epochs, training is stopped. The loss curves and AUC curves of the two models are shown in the Figure 12. It can be seen that the training loss of the DIN model continues to decrease, while its validation loss (based on fixed-weight loss) tends to stabilize after about 80 epochs and even slightly rebounds—indicating the model begins to over-fit, as it continues to optimize on training set samples but no longer improves generalization on the validation set. Early stopping for DIN occurs at around 100 epochs; continuing training would further increase validation loss and exacerbate over-fitting, so this strategy is reasonable. Nevertheless, the gap between its validation and training loss remains obvious, suggesting limited generalization ability due to the lack of loss function optimization.

For the DIN+DIM model, the optimized dynamic weight loss function drives the overall loss to decrease more significantly—especially the validation loss, which is significantly lower than that of the original DIN. This shows the model performs better on the validation set after loss function optimization and DIM introduction. The training loss and validation loss curves are closer, demonstrating enhanced generalization ability and alleviated over-fitting compared with the original DIN. Moreover, the validation loss of DIN+DIM decreases and stabilizes after about 120 epochs, a longer effective training period than the original DIN. This is because the dynamic weight loss function avoids premature convergence to local optima caused by over-emphasizing non-critical samples, allowing the model to maintain effective learning for longer.

On the whole, DIN+DIM is superior to the original DIN in both loss (driven by optimized loss function) and AUC, and the early stopping strategy (based on optimized validation loss) is reasonable. Subsequent experiments will be conducted based on the current early stopping parameters and the optimized loss function to ensure the consistency and stability of the comparative experiments. During the experiment, the comparison results of different experimental performances are shown in

Table 5. Table of performance comparison of different models.

	DeepFM	WideDeep	DCN	NFM	AFM	DIN	AutoInt	DIN+DIM
Precision	0.6734	0.5617	0.5876	0.7931	0.7237	0.6786	0.6804	0.8679
Recall	0.5352	0.4671	0.4141	0.6851	0.6978	0.6530	0.5721	0.9067
F1 Score	0.5964	0.5101	0.4858	0.7351	0.7105	0.6656	0.6216	0.8878
AUC	0.7145	0.5664	0.6531	0.8381	0.8197	0.7650	0.6724	0.9607
AP	0.6335	0.5151	0.5874	0.7709	0.793	0.7247	0.6344	0.9537
Hit Ratio	0.4501	0.4723	0.4028	0.5041	0.5543	0.5177	0.4893	0.5200
NDCG	0.5604	0.6383	0.5341	0.8335	0.9522	0.4915	0.7236	0.6432
MAP	0.4501	0.4723	0.4028	0.5041	0.5543	0.4933	0.4893	0.5253
MRR	0.4501	0.4723	0.4028	0.5041	0.5543	0.4933	0.4893	0.5200

, and the settings of relevant hyperparameters are presented in

Table 6. Training hyperparameter settings.

Symbol Meaning	Symbol Representation	Set Value
Learning rate	Learning Rate	1 × 10−4
Optimizer	Optimizer	Adam
Batch size	Batch Size	64
Number of iterations	Epochs	100

.

To verify the performance improvement of the model in the network security operation playbook recommendation task after introducing dynamic interest modeling, this paper compares and analyzes the performance of various evaluation metrics between the original DIN model and the improved model (DIN+DIM) with dynamic interest modeling introduced. The results are shown in the

Table 7. Table of performance comparison between recommendation network based on dynamic interest modeling and DIN model.

	DIN	DIN+DIM
Precision	0.6786	0.8679
Recall	0.6530	0.9067
F1 Score	0.6656	0.8878
AUC	0.7650	0.9607
AP	0.7247	0.9537
Hit Ratio	0.5177	0.5200
NDCG	0.4915	0.6432
MAP	0.4933	0.5253
MRR	0.4933	0.5200

:

According to the tabulated results, the introduction of DIM leads to a substantial improvement in recommendation performance. Precision increases significantly from 0.6786 to 0.8679, yielding an improvement of approximately 27%. Recall improves from 0.6530 to 0.9067, representing a relative gain of about 39%. Similarly, the F1 Score rises from 0.6656 to 0.8878 (≈33% improvement), while AUC improves from 0.7650 to 0.9607 (≈26% improvement). Moreover, AP increases from 0.7247 to 0.9537 (≈31% improvement), and NDCG improves from 0.4915 to 0.6432 (≈31% improvement). In contrast, the improvements in Hit Ratio, MAP, and MRR are relatively marginal, showing only minor gains.

These results demonstrate that the integration of DIM into DIN substantially enhances the accuracy of recommending relevant alert–playbook pairs. The enhanced precision indicates that the model is more effective at filtering out irrelevant playbooks while retaining those that are truly relevant, thereby improving the quality of recommendations. The significant improvement in AUC highlights the model’s strengthened ability to discriminate between positive and negative samples, which contributes to more accurate identification of alert–playbook correlations and, consequently, to more reliable security incident prediction.

The observed gains can be attributed to DIM’s ability to accurately capture the dynamic shifts in analysts’ interests toward security alerts. As a result, the recommendation system achieves not only improved matching performance but also better ranking quality. The increases in AP and NDCG reveal that the enhanced model not only recommends a higher number of relevant playbooks but also prioritizes the most relevant ones at the top of the ranked list, thereby improving both system effectiveness and analyst experience. For alert response tasks in network security operations, such dynamic interest modeling provides more precise and actionable recommendations, enabling analysts to select the most effective mitigation strategies.

Although the improvements in MAP and MRR are relatively modest, the significant performance gains across other metrics provide compelling evidence that DIN+DIM outperforms the baseline DIN, particularly in complex security scenarios where the dynamic evolution of analysts’ interests must be effectively modeled.

To provide a more intuitive comparison, the performance improvements across Precision, Recall, F1 Score, AUC, AP, Hit Ratio, NDCG, MAP, and MRR are illustrated in the following figure.

As shown in Figure 13, DIN+DIM outperforms DIN across most performance metrics, with particularly notable improvements in Precision, Recall, F1 Score, AUC, and AP. These results indicate that incorporating dynamic interest modeling substantially enhances the system’s accuracy, recall ability, and ranking quality. Such improvements enable the recommendation system to better adapt to the evolving needs of users, providing more precise and contextually relevant playbook recommendations, especially in complex network security scenarios.

Overall, the integration of DIM into DIN yields significant performance gains, particularly in Precision, Recall, AUC, and AP, demonstrating that DIN+DIM can more effectively capture and adapt to the dynamic shifts in analysts’ interests. By modeling dynamic user interests, the recommendation system achieves markedly improved playbook recommendation quality, exhibiting stronger recommendation capability. This advantage makes the approach especially suitable for security domains where timeliness and accuracy are critical requirements.

6. Conclusions and Future Work

To address the limitation of traditional recommendation methods in security operations—namely, their inability to effectively capture the dynamic evolution of alert interests—this paper proposes a dynamic interest modeling-based recommendation approach. Building on the classical DIN framework, the method incorporates a Transformer encoder and an LSTM model to separately model long-term and short-term interests, thereby enhancing the model’s adaptability and performance in dynamic security operation environments. Experimental results demonstrate significant improvements in key metrics such as Precision and Recall, while also alleviating the issue of model over-fitting. Overall, the proposed dynamic interest modeling approach not only improves the accuracy and adaptability of security playbook recommendations, but also provides new technical insights and practical experience for advancing intelligent recommendation systems in security operations.

In the future, our research will explore more efficient computing or lightweight architectures, integrate multi-modal data sources, and enhance the accuracy of recommendations. We will further adopt self-supervised or reinforcement learning methods to optimize recommendation strategies for adapting to dynamically changing security threats.