Search Results (17)

Industrial Control Systems (ICS) are increasingly targeted by sophisticated and evolving cyberattacks, while conventional static defense mechanisms and isolated intrusion detection models often lack the robustness required to cope with such dynamic threats. To overcome these limitations, we propose RHAD (Reinforced Heterogeneous Anomaly Detector), a resilient and adaptive anomaly detection framework specifically designed for ICS environments. RHAD combines a heterogeneous ensemble of detection models with a confidence-aware scheduling mechanism guided by reinforcement learning (RL), alongside a time-decaying sliding window voting strategy to enhance detection accuracy and temporal robustness. The proposed architecture establishes a modular collaborative framework that enables dynamic and fine-grained protection for industrial network traffic. At its core, the RL-based scheduler leverages the Proximal Policy Optimization (PPO) algorithm to dynamically assign model weights and orchestrate container-level executor replacement in real time, driven by network state observations and runtime performance feedback. We evaluate RHAD using two publicly available ICS datasets—SCADA and WDT—achieving 99.19% accuracy with an F1-score of 0.989 on SCADA, and 98.35% accuracy with an F1-score of 0.987 on WDT. These results significantly outperform state-of-the-art deep learning baselines, confirming RHAD’s robustness under class imbalance conditions. Thus, RHAD provides a promising foundation for resilient ICS security and shows strong potential for broader deployment in cyber-physical systems. Full article

The increasing integration of photovoltaic (PV) systems into smart grids introduces new cybersecurity vulnerabilities, particularly against cyber-physical attacks that can manipulate grid operations and disrupt renewable energy generation. This paper proposes a multi-layered cyber-resilient PV optimization framework, leveraging digital twin-based deception, reinforcement learning-driven cyber defense, and blockchain authentication to enhance grid security and operational efficiency. A deceptive cyber-defense mechanism is developed using digital twin technology to mislead adversaries, dynamically generating synthetic PV operational data to divert attack focus away from real assets. A deep reinforcement learning (DRL)-based defense model optimizes adaptive attack mitigation strategies, ensuring real-time response to evolving cyber threats. Blockchain authentication is incorporated to prevent unauthorized data manipulation and secure system integrity. The proposed framework is modeled as a multi-objective optimization problem, balancing attack diversion efficiency, system resilience, computational overhead, and energy dispatch efficiency. A non-dominated sorting genetic algorithm (NSGA-III) is employed to achieve Pareto-optimal solutions, ensuring high system resilience while minimizing computational burdens. Extensive case studies on a realistic PV-integrated smart grid test system demonstrate that the framework achieves an attack diversion efficiency of up to 94.2%, improves cyberattack detection rates to 98.5%, and maintains an energy dispatch efficiency above 96.2%, even under coordinated cyber threats. Furthermore, computational overhead is analyzed to ensure that security interventions do not impose excessive delays on grid operation. The results validate that digital twin-based deception, reinforcement learning, and blockchain authentication can significantly enhance cyber-resilience in PV-integrated smart grids. This research provides a scalable and adaptive cybersecurity framework that can be applied to future renewable energy systems, ensuring grid security, operational stability, and sustainable energy management under adversarial conditions. Full article

Faced with challenges posed by sophisticated cyber attacks and dynamic characteristics of cyberspace, the autonomous cyber defense (ACD) technology has shown its effectiveness. However, traditional decision-making methods for ACD are unable to effectively characterize the network topology and internode dependencies, which makes it difficult for defenders to identify key nodes and critical attack paths. Therefore, this paper proposes an enhanced decision-making method combining graph embedding with reinforcement learning algorithms. By constructing a game model for cyber confrontations, this paper models important elements of the network topology for decision-making, which guide the defender to dynamically optimize its strategy based on topology awareness. We improve the reinforcement learning with the Node2vec algorithm to characterize information for the defender from the network. And, node attributes and network structural features are embedded into low-dimensional vectors instead of using traditional one-hot encoding, which can address the perceptual bottleneck in high-dimensional sparse environments. Meanwhile, the algorithm training environment Cyberwheel is extended by adding new fine-grained defense mechanisms to enhance the utility and portability of ACD. In experiments, our decision-making method based on graph embedding is compared and analyzed with traditional perception methods. The results show and verify the superior performance of our approach in the strategy selection of defensive decision-making. Also, diverse parameters of the graph representation model Node2vec are analyzed and compared to find the impact on the enhancement of the embedding effectiveness for the decision-making of ACD. Full article

In response to the rising volume and sophistication of cyber intrusions, data-oriented methods have emerged as critical defensive measures. While machine learning—including neural network-based solutions—has demonstrated strong capabilities in identifying malicious activities, several fundamental challenges remain. Chief among these difficulties are the substantial resource demands related to data preprocessing and inference procedures, limited scalability beyond centralized environments, and the necessity of deploying multiple specialized detection models to address diverse stages of the cyber kill chain. This paper introduces a contextual bandit-based reinforcement learning approach, designed to reduce operational expenditures and enhance detection cost-efficiency by introducing an adaptive decision boundary within a layered detection scheme. The proposed framework continually measures the confidence of each participating detection model, applying a reward-driven mechanism to balance cost and accuracy. Specifically, each potential action, representing a particular decision boundary, earns a reward reflecting its overall cost-to-effectiveness ratio, thereby prioritizing reduced overheads. We validated our method using two highly representative datasets that capture prevalent modern-day threats: phishing and malware. Our findings show that this contextual bandit-based strategy adeptly regulates the frequency of resource-intensive detection tasks, significantly lowering both inference and processing expenses. Remarkably, it achieves this reduction with minimal compromise to overall detection accuracy and efficacy. Full article

With the widespread adoption of wireless communication technologies in modern high-speed rail systems, the Train-to-Ground (T2G) communication system for Electric/Diesel Multiple Units (EMU/DMU) has become essential for train operation monitoring and fault diagnosis. However, this system is increasingly vulnerable to various cyber-physical threats, necessitating more intelligent and adaptive security protection mechanisms. This paper presents an intelligent security defense framework that integrates intrusion detection, risk scoring, and response mechanisms to enhance the security and responsiveness of the T2G communication system. First, feature selection is performed on the TON_IoT dataset to develop a Dream Optimization Algorithm (DOA)-optimized backpropagation neural network (DOA-BPNN) model for efficient anomaly detection. A Bayesian risk scoring module then quantifies detection outcomes and classifies risk levels, improving threat detection accuracy. Finally, a Q-learning-based reinforcement learning (RL) module dynamically selects optimal defense actions based on identified risk levels and attack patterns to mitigate system threats. Experimental results demonstrate improved performance in both multi-class and binary classification tasks compared to conventional methods. The implementation of the Bayesian risk scoring and decision-making modules leads to a 63.56% reduction in system risk scores, confirming the effectiveness and robustness of the proposed approach in an experimental environment. Full article

Network security and intrusion detection and response (IDR) are necessary issues nowadays. Enhancing our cyber defense by discovering advanced machine learning models, such as reinforcement learning and Q-learning, is a crucial security measure. This study proposes a novel intrusion response method by implementing an off-policy Q-learning approach. We test the validity of our model by conducting a goodness-of-fit analysis and proving its efficiency. By performing sensitivity analysis, we prove that it is possible to protect our network successfully and establish an immediate response mechanism that could be successfully implemented in intrusion response (IR) systems. Full article

Sophisticated mechanisms for attacking computer networks are emerging, making it crucial to have equally advanced mechanisms in place to defend against these malicious attacks. Autonomous cyber operations (ACOs) are considered a potential solution to provide timely defense. In ACOs, an agent that attacks the network is called a red agent, while an agent that defends against the red agent is called a blue agent. In real-world scenarios, different types of red agents can attack a network, requiring blue agents to defend against a variety of red agents, each with unique attack strategies and goals. This requires the training of blue agents capable of responding effectively, regardless of the specific strategy employed RED. Additionally, a generic blue agent must also be adaptable to different network topologies. This paper presents a framework for the training of a generic blue agent capable of defending against various red agents. The framework combines reinforcement learning (RL) and supervised learning. RL is used to train a blue agent against a specific red agent in a specific networking environment, resulting in multiple RL-trained blue agents—one for each red agent. Supervised learning is then used to train a generic blue agent using these RL-trained blue agents. Our results demonstrate that the proposed framework successfully trains a generic blue agent that can defend against different types of red agents across various network topologies. The framework demonstrates consistently improved performance over a range of existing methods, as validated through extensive empirical evaluation. Detailed comparisons highlight its robustness and generalization capabilities. Additionally, to enable generalization across different adversarial strategies, the framework employs a variational autoencoder (VAE) that learns compact latent representations of observations, allowing the blue agent to focus on high-level behavioral features rather than raw inputs. Our results demonstrate that incorporating a VAE into the proposed framework further improves its overall performance. Full article

Recently, the IT industry has become larger, and cloud service has rapidly increased; thus cybersecurity to protect sensitive data from attacks has become an important factor. However, cloud services have become larger, making the surface area larger, and a complex cyber environment leads to difficulty managing and defending. With the rise of artificial intelligence, applying artificial intelligence to a cyber environment to automatically detect and respond to cyberattacks has begun to get attention. In order to apply artificial intelligence in cyber environments, a simulation framework that is easily applicable and can represent real situations well is needed. In this study, we introduce the framework Cyber Environment (CYE) that provides useful components that abstract complex and large cloud environments. Additionally, we use CYE to reproduce real-world situations into the scenario and apply reinforcement learning for training automated intelligence defense agents. Full article

The increasing sophistication and frequency of cyber attacks necessitate automated and intelligent response mechanisms that can adapt to evolving threats. This paper presents ARCS (Adaptive Reinforcement learning for Cybersecurity Strategy), a novel framework that leverages deep reinforcement learning to optimize automated incident response strategies in cybersecurity systems. Our approach uniquely combines state representation learning of security events with a hierarchical decision-making process to map attack patterns to optimal defense measures. The framework employs a custom reward mechanism that balances incident resolution time, system stability, and defense effectiveness. Using a comprehensive dataset of 20,000 cybersecurity incidents, we demonstrate that ARCS achieves 27.3% faster incident resolution times and 31.2% higher defense effectiveness compared to traditional rule-based approaches. The framework shows particular strength in handling complex, multi-stage attacks, reducing false positive rates by 42.8% while maintaining robust system performance. Through extensive experiments, we validated that our approach can effectively generalize across different attack types and adapt to previously unseen threat patterns. The results suggest that reinforcement learning-based automation can significantly enhance cybersecurity incident response capabilities, particularly in environments requiring rapid and precise defensive actions. Full article

In light of the escalating complexity of the cyber threat environment, the role of Collaborative Intrusion Detection Systems (CIDSs) in reinforcing contemporary cybersecurity defenses is becoming ever more critical. This paper presents a Blockchain-based Collaborative Intrusion Detection Framework (BCIDF), an innovative methodology aimed at enhancing the efficacy of threat detection and information dissemination. To address the issue of alert collisions during data exchange, an Alternating Random Assignment Selection Mechanism (ARASM) is proposed. This mechanism aims to optimize the selection process of domain leader nodes, thereby partitioning traffic and reducing the size of conflict domains. Unlike conventional CIDS approaches that typically rely on independent node-level detection, our framework incorporates a Weighted Random Forest (WRF) ensemble learning algorithm, enabling collaborative detection among nodes and significantly boosting the system’s overall detection capability. The viability of the BCIDF framework has been rigorously assessed through extensive experimentation utilizing the NSL-KDD dataset. The empirical findings indicate that BCIDF outperforms traditional intrusion detection systems in terms of detection precision, offering a robust and highly effective solution within the realm of cybersecurity. Full article

In the current landscape where cybersecurity threats are escalating in complexity and frequency, traditional defense mechanisms like rule-based firewalls and signature-based detection are proving inadequate. The dynamism and sophistication of modern cyber-attacks necessitate advanced solutions that can evolve and adapt in real-time. Enter the field of deep reinforcement learning (DRL), a branch of artificial intelligence that has been effectively tackling complex decision-making problems across various domains, including cybersecurity. In this study, we advance the field by implementing a DRL framework to simulate cyber-attacks, drawing on authentic scenarios to enhance the realism and applicability of the simulations. By meticulously adapting DRL algorithms to the nuanced requirements of cybersecurity contexts—such as custom reward structures and actions, adversarial training, and dynamic environments—we provide a tailored approach that significantly improves upon traditional methods. Our research undertakes a thorough comparative analysis of three sophisticated DRL algorithms—deep Q-network (DQN), actor–critic, and proximal policy optimization (PPO)—against the traditional RL algorithm Q-learning, within a controlled simulation environment reflective of real-world cyber threats. The findings are striking: the actor–critic algorithm not only outperformed its counterparts with a success rate of 0.78 but also demonstrated superior efficiency, requiring the fewest iterations (171) to complete an episode and achieving the highest average reward of 4.8. In comparison, DQN, PPO, and Q-learning lagged slightly behind. These results underscore the critical impact of selecting the most fitting algorithm for cybersecurity simulations, as the right choice leads to more effective learning and defense strategies. The impressive performance of the actor–critic algorithm in this study marks a significant stride towards the development of adaptive, intelligent cybersecurity systems capable of countering the increasingly sophisticated landscape of cyber threats. Our study not only contributes a robust model for simulating cyber threats but also provides a scalable framework that can be adapted to various cybersecurity challenges. Full article

In the face of an increasingly intricate network structure and a multitude of security threats, cyber deception defenders often employ deception assets to safeguard critical real assets. However, when it comes to the intranet lateral movement attackers in the cyber kill chain, the deployment of deception assets confronts the challenges of lack of dynamics, inability to make real-time decisions, and not considering the dynamic change of an attacker’s strategy. To address these issues, this study introduces a novel maze pathfinding model tailored to the lateral movement context, in which we try to find out the attacker’s location to deploy deception assets accurately for interception. The attack–defense process is modeled as a multi-agent stochastic game, by comparing it with random action policy and Minimax-Q algorithm, we choose Nash Q-learning to solve the deception asset’s deployment strategy to achieve the optimal solution effect. Extensive simulation tests reveal that our proposed model exhibits good convergence properties. Moreover, the average defense success rate surpasses 70%, attesting to the model’s efficacy. Full article

Technological advancement of vehicle platforms exposes opportunities for new attack paths and vulnerabilities. Static cyber defenses can help mitigate certain attacks, but those attacks must generally be known ahead of time, and the cyber defenses must be hand-crafted by experts. This research explores reinforcement learning (RL) as a path to achieve autonomous, intelligent cyber defense of vehicle control networks—namely, the controller area network (CAN) bus. We train an RL agent for the CAN bus using Toyota’s Portable Automotive Security Testbed with Adaptability (PASTA). We then apply the U.S. Army Combat Capabilities Development Command (DEVCOM) Army Research Laboratory’s methodology for quantitative measurement of cyber resilience to assess the agent’s effect on the vehicle testbed in a contested cyberspace environment. Despite all defenses having similar traditional performance measures, our RL agent averaged a 90% cyber resilience measurement during drive cycles executed on hardware versus 41% for a naïve static timing defense and 98% for the bespoke timing-based defense. Our results also show that an RL-based agent can detect and block injection attacks on a vehicle CAN bus in a laboratory environment with greater cyber resilience than prior learning approaches (1% for convolutional networks and 0% for recurrent networks). With further research, we believe there is potential for using RL in the autonomous intelligent cyber defense agent concept. Full article

In the rapidly evolving landscape of internet usage, ensuring robust cybersecurity measures has become a paramount concern across diverse fields. Among the numerous cyber threats, denial of service (DoS) and distributed denial of service (DDoS) attacks pose significant risks, as they can render websites and servers inaccessible to their intended users. Conventional intrusion detection methods encounter substantial challenges in effectively identifying and mitigating these attacks due to their widespread nature, intricate patterns, and computational complexities. However, by harnessing the power of deep learning-based techniques, our proposed dense channel-spatial attention model exhibits exceptional accuracy in detecting and classifying DoS and DDoS attacks. The successful implementation of our proposed framework addresses the challenges posed by imbalanced data and exhibits its potential for real-world applications. By leveraging the dense channel-spatial attention mechanism, our model can precisely identify and classify DoS and DDoS attacks, bolstering the cybersecurity defenses of websites and servers. The high accuracy rates achieved across different datasets reinforce the robustness of our approach, underscoring its efficacy in enhancing intrusion detection capabilities. As a result, our framework holds promise in bolstering cybersecurity measures in real-world scenarios, contributing to the ongoing efforts to safeguard against cyber threats in an increasingly interconnected digital landscape. Comparative analysis with current intrusion detection methods reveals the superior performance of our model. We achieved accuracy rates of 99.38%, 99.26%, and 99.43% for Bot-IoT, CICIDS2017, and UNSW_NB15 datasets, respectively. These remarkable results demonstrate the capability of our approach to accurately detect and classify various types of DoS and DDoS assaults. By leveraging the inherent strengths of deep learning, such as pattern recognition and feature extraction, our model effectively overcomes the limitations of traditional methods, enhancing the accuracy and efficiency of intrusion detection systems. Full article

This paper addressed the optimal policy selection problem of attacker and sensor in cyber-physical systems (CPSs) under denial of service (DoS) attacks. Since the sensor and the attacker have opposite goals, a two-player zero-sum game is introduced to describe the game between the sensor and the attacker, and the Nash equilibrium strategies are studied to obtain the optimal actions. In order to effectively evaluate and quantify the gains, a reinforcement learning algorithm is proposed to dynamically adjust the corresponding strategies. Furthermore, security state estimation is introduced to evaluate the impact of offensive and defensive strategies on CPSs. In the algorithm, the $\epsilon$-greedy policy is improved to make optimal choices based on sufficient learning, achieving a balance of exploration and exploitation. It is worth noting that the channel reliability factor is considered in order to study CPSs with multiple reasons for packet loss. The reinforcement learning algorithm is designed in two scenarios: reliable channel (that is, the reason for packet loss is only DoS attacks) and unreliable channel (the reason for packet loss is not entirely from DoS attacks). The simulation results of the two scenarios show that the proposed reinforcement learning algorithm can quickly converge to the Nash equilibrium policies of both sides, proving the availability and effectiveness of the algorithm. Full article