Infrastructure and Tools for Testing the Vulnerability of Control Systems to Cyberattacks: A Coal Mine Industrial Facility Case

Abstract

Testing the vulnerability of information systems to cyberattacks is essential to ensure the operational security of organizations and industrial processes. In particular, it is essential to ensure the resilience of industrial processes, as a possible cyberattack can lead to process malfunctions and even process shutdowns, which can lead to substantial economic losses. The possibility of various attacks, e.g., ransomware, phishing, or advanced persistent threats (APTs), requires the evaluation of the effectiveness of cyberattack detection and incident response mechanisms. In industry, it is often impossible to carry out this type of test without risking system disruption, making it difficult to assess the true effectiveness of security features. This article discusses the issues concerned with testing the cyber resilience of a system operating in a real coal mine. First, this work briefly presents the hardware and software architecture used in the coal mine. Secondly, it describes the problem of replicating a real system in the laboratory and the necessary tools and methods used to implement a resilient system architecture. Finally, the scenarios of cyberattacks are detailed, and the obtained results are discussed.

1. Introduction

In recent years, we have seen a significant and increasing number of cyberattacks, posing a serious threat to private users, organizations, and industrial systems [1,2]. This problem affects not only information technology (IT) systems but also automated production lines that rely on operational technology (OT) systems. Advanced industrial OT systems enable efficient process control, visualization, and database systems, e.g., supervisory control and data acquisition (SCADA) systems, which result in technologically efficient and economically viable process operations. Since all the components of production lines are connected, the risk of cyberattacks is significant and an increasing threat to the industrial networks used in industries such as energy, robotics, and automotive [3,4,5,6,7,8,9,10]. Numerous cyberattack methods are possible, e.g., ransomware [11,12], malware [13,14], phishing [15,16] or distributed denial-of-service (DDoS) attacks [17,18]. In IT systems, cyberattacks can lead to important data leakage, which is likely to entail a loss of credibility and customer trust. In industrial OT systems, cyberattacks can result in process disturbances, malfunctions, or even shutdowns. Unfortunately, the consequences of cyberattacks on industrial OT systems may result in high economic losses. While process malfunctions typically decrease product quality [19], leading to lower economic profits, shutdowns require significant expenses to restore normal process operations [20]. As we have observed an increasing number of cyberattacks on OT systems, efficient detection methods and techniques capable of limiting the consequences of such attacks are essential [21].

Cyberattacks on critical installations such as power plants, energy networks, wastewater treatment plants, water supply networks, transportation systems, and telecommunication infrastructure are particularly dangerous. Aside from the significant financial consequences, the social and physiological implications for individuals and society are also of great importance [22]. This is because critical installations are fundamental to the functioning of societies. Unfortunately, their disruption can lead to chaos, endanger people’s lives, and badly affect the economy. Let us highlight the following two examples of cyberattacks on critical installations: the Stuxnet virus incident, which attacked industrial control systems in Iran [23], as well as the 2015 cyberattack on the power grid in Ukraine, which led to power outages [24]. Therefore, protecting critical installations is the highest priority for both the public and private sectors [25]. Unfortunately, the risk of cyberattacks on critical infrastructure is very high [26].

Mines are an important component of critical infrastructure as they supply natural resources such as coal, metal ores, or minerals that are crucial to many industries. Raw materials from mines are necessary in the energy sector, heavy industry, and high-tech manufacturing. Therefore, the security of mines is important for the stability of the economy and the continuity of other industrial sectors [27]. Automatic control and monitoring systems, increasingly popular in mines, can become the target of cyberattacks. For this reason, protecting mines as critical infrastructure facilities requires the appropriate security measures, both physical and software [28]. Any disruption or interruption to the normal operation of mines can have far-reaching effects, e.g., shortages of raw materials, increased energy prices, production problems in various industry sectors, and disruptions in supply chains. Energy supply problems are particularly dangerous as they may decrease energy security [29]. The problem is particularly relevant for countries whose economies depend strongly on extracting raw materials [30]. Ensuring the stable operation of mines is critical for day-to-day economic activity and national security, prioritizing it in critical infrastructure protection planning [31].

Let us emphasize that in the case of critical facilities such as power plants, water networks, mines, or transport systems, thorough real cyberattacks on the real working infrastructure meant to evaluate its resilience are practically impossible since they create too much risk, as any disruption could have serious consequences for the operation of the entire system [32]. Therefore, the effectiveness of cyberattack detection and mitigation mechanisms can only be fully verified under laboratory conditions [33]. Experiments carried out in laboratories allow us to assess the efficiency of cyberattack detection and mitigation systems, considering a very rich set of scenarios for the simulated attacks that are too dangerous in practice. Such laboratory experiments make it possible to verify the efficiency of existing protection systems and develop new methods when necessary [34]. We have to stress that although tests in laboratories are very useful, they cannot completely replace monitoring for real attacks in actual applications. However, laboratory tests provide an opportunity to safely verify if and to what extent the existing or newly developed security measures are effective before they are applied to the actual critical systems. Thanks to such an approach, the risk associated with implementing new solutions can be minimized.

The laboratory testing methodology used is critical from the point of view of developing efficient cyberattack monitoring and mitigation methods. In particular, we must ensure that the test laboratory environment correctly replicates the necessary infrastructure, which is often difficult and costly [35]. When this issue poses problems, the solution may be to replicate the process in a simplified manner, e.g., using a much less complex hardware environment [36], but with the objective of obtaining conditions as close as possible to those of the actual installation. Such an approach would be much cheaper than replicating the entire system architecture in a laboratory environment.

The article discusses the issues concerned with testing the cyber resilience of a system operating in a coal mine located in Poland. It describes the problem of replicating a real system in the laboratory, as well as the necessary tools and methods used to implement a resilient system architecture. In particular, we focus on how to use the software tools appropriately and modify network traffic to simulate the most likely cyberattacks. The proposed approach is based directly on network traffic, which creates opportunities for a full spectrum of testing and not just the vulnerability evaluation of automatic control algorithms [34]. The key is to use the traffic from the real site for which the solution is being developed, an approach that allows for the preparation of cybersecurity methods and their reliable testing without disrupting the target industrial system. It is worth noting that conducting simulated attacks directed at a critical infrastructure’s industrial environment is unacceptable. Operating on network traffic allows for the simulation of the following tests: penetration, resilience, configuration, and system scanning for known vulnerabilities. The proposed solution is designed to be generic and flexible, so it can be applied in different contexts without significant modifications. This approach enables greater scalability, reuse, and adaptation to new needs and requirements.

This article is structured in the following way. First, Section 2 describes the IT/OT infrastructure in the coal mine. Section 3 deals with the laboratory infrastructure, while Section 4 presents the software tools used to record, modify, and replay the network traffic, and Section 5 defines cyberattacks’ scenarios and discusses the obtained results. Finally, Section 6 summarizes and concludes the article.

2. IT/OT Infrastructure in the Coal Mine

The IT/OT infrastructure is key in managing mining processes, monitoring safety, and optimizing production. Modern systems use advanced IT and OT systems to carry out the precise management of resources, automation of operations, and remote monitoring of machine status and environmental conditions. Figure 1 depicts a typical three-layer industrial control infrastructure in which each layer serves a specific purpose and interacts with other layers.

The top layer (or supervisory layer) sets goals, defines strategies, and allocates resources. Information is considered at a high level, without the implementation details. This layer contains the management software responsible for overseeing production schedules, allocating resources, and monitoring performance. Goals and strategies are sent to the intermediate layer, where they are translated into specific tasks and activities. The purpose of the intermediate layer is to coordinate and optimize the performance of the lower-level components. The lower layer (or execution layer) performs specific tasks and actions at the physical level of the lower-level components or subsystems of the system, typically hardware devices, actuators, sensors.

The three-layer control structure provides a modular and scalable approach to system design. It is widely used in various fields where complex systems must be controlled and managed efficiently. A good example would be mines, which have highly complex information exchange systems.

The considered IT/OT infrastructure used in mines is shown in Figure 2. It is based on the general three-layer industrial control infrastructure shown in Figure 1, with some necessary modifications [37,38]. Let us enumerate and discuss the main elements of the infrastructure, as follows:

• The SCADA systems monitor and control all industrial processes in the mine facility [39]. Additionally, supervision of the entire technical infrastructure and all its elements is performed, both on the surface and underground. Process operators remotely monitor crucial parameters and signal values using the SCADA systems. Furthermore, the condition of the equipment and machinery components is constantly monitored. This helps preserve safe working conditions and prevent failures.
• Communication networks include wired (e.g., fiber optics) and wireless (Wi-Fi, LTE, 5G, radio) technologies [40,41,42,43]. Specialized wireless networks are typically required underground since they can resist harsh conditions, e.g., high humidity, dust, and vibration. Such networks enable communication between the operators, remote control of the machinery, and high-speed transmission of the data from various sensors and measuring devices.
• An array of sensors deployed in different parts of the mine provide current values of the key variables, e.g., temperature, methane concentration, oxygen levels, air pressure, and vibration [44,45]. The obtained data samples are monitored online. In addition to these classical measurements, video monitoring systems comprised of cameras allow for the observation of machinery operation and mining processes and help us to identify potentially dangerous situations, e.g., landslides. Contemporary mines may also use drones to monitor the situation efficiently.
• Integrated manufacturing execution systems (MESs) and enterprise resource planning (ERP) systems [46,47,48,49] have the objective of integrating data from different sources. These tools optimize the use of resources and processes, leading to cost minimization. The ERP systems are pivotal in coordinating all interactions between the technical departments and management. This would likely make rapid decision-making possible.
• Automatic control systems, which can operate without direct human intervention, will eliminate the need for humans in remote and dangerous environments and replace humans when fast reactions are necessary and/or impossible for humans [50,51,52]. In addition to typical process control systems, e.g., stabilization of the key variables, we must mention remotely operated vehicles, automatic transportation systems, and robots. The widespread utilization of advanced Internet of Things (IoT) technology and artificial intelligence allows for the coordination and control of all mentioned devices automatically and remotely [37,50,53].
• Advanced monitoring and alarm systems will provide the immediate notification of hazards and ensure safety for employees. They include devices used to constantly monitor the pivotal parameters of the environment, e.g., unacceptable methane concentration in the air or rising water levels. Additionally, the likely collapse of wall and ceiling supports can be signaled [53,54,55].

In addition to being connected with each other, the consecutive components of the system may send information externally, which greatly increases vulnerability to cyberattacks. If key IT/OT network elements are attacked, automatic and remote equipment control and the online monitoring of current conditions may be impossible. For example, a cyberattack on the SCADA system, the communications network, automatic control units, or monitoring, security, and alarm systems is likely to badly disrupt the mine’s operation. Staff health and life can be in danger within the most critical situation. On the one hand, the increasing use of interconnected network systems, as well as automatic control and monitoring units, is very beneficial for optimal mine operation. On the other hand, unfortunately, the more complex the IT/OT infrastructure, the higher the risk of cyberattacks.

In the mine considered in this work, information in the OT network is mainly exchanged over the Modbus TCP protocol. Hence, the research focuses on this protocol. However, the software testing and development approach presented next is universal and can be successfully applied to other TCP/IP-based communication protocols such as BACNet/IP, S7, Profinet or DNP3.

3. Laboratory Infrastructure

The laboratory infrastructure used to check and evaluate the effectiveness of security features and the operation of systems in critical installations must reflect the real-world conditions in these facilities as closely as possible. Thanks to that, the test results will likely be reliable. Furthermore, assessing the performance of various solutions in a real operational environment is possible. We must stress that laboratory tests conducted under near-real conditions can show how the systems respond to different cyberattacks. Additionally, we can assess the consequences of possible software and hardware failures. Replicating the real process in the laboratory environment is essential for the effectiveness of the whole study. We aim to have a laboratory system whose behavior is very similar to that of the real process. In fact, the more similar the behavior of the laboratory system to the actual one, the higher the likelihood of detecting potential security vulnerabilities. For example, simulating the actual load on the IT/OT infrastructure, network, or the behavior of industrial equipment in the laboratory allows for the assessment of how systems respond to abnormal situations, such as overloads or failures which are possible in the mine but we are not typically allowed to cause within the real industrial process.

A well-designed and implemented laboratory infrastructure provides predictability of the system behavior. It allows us to see how cyber threat detection and mitigation systems perform considering real or close-to-real industrial plant operation conditions. Thanks to that, we can assess whether and to what extent the designed systems can operate normally or close to normal in the event of a failure or emergency. As a result, we can assess whether the SCADA systems controlling industrial processes are resilient to simulated cyberattack attempts. Observing the laboratory process behavior greatly helps us to develop adequate detection methods and mitigation approaches.

In addition to assessing the performance of existing solutions under abnormal situations, e.g., cyberattacks, laboratory tests help validate new solutions. Obviously, it is much safer, cheaper, and faster to conduct experiments with new algorithms and software and hardware tools in a fully controlled laboratory environment than in the real infrastructure. We can detect general misconceptions in the new approach and implementation errors thanks to such an approach. Consequently, when the new technologies are implemented in the actual industrial facility, we believe it will work as expected and greatly reduce the risk of problems. Finally, the consequences of abnormal situations, e.g., failures or cyberattacks, can be removed quickly and do not require financial expenses as in the real industrial setup. In the actual IT/OT infrastructure, the results of cyberattacks may require process reconfiguration, hardware or software alterations, or even shutdown.

A well-designed and implemented test laboratory environment that mimics the behavior of the actual system can also be used to train process operators and engineers. In particular, the laboratory infrastructure allows for the evaluation of the efficiency of the detection and mitigation methods in a safe environment. We can consider not only the abnormal situations that have occurred in the past or are predicted to happen but also completely new ones that have never been experienced in the industrial setting.

As discussed above, having a realistic test environment has many benefits. On the other hand, we must emphasize that properly designing and implementing a laboratory infrastructure that closely mimics the real industrial setup may be complicated, time-consuming, and expensive. In our case, replicating the full infrastructure of a mine in the laboratory required the technology, equipment, and control systems similar to those of a real facility. Notably, perfect or close-to-ideal replication required great attention to hardware and software to accurately simulate a real plant’s operation. Since we were unable to perform tests in the actual coal mine setting, laboratory tests offer the only possibility to attempt to evaluate the system’s resilience to a cyberattack. However, it should be noted that the lack of actual testing in a real mine environment means that the complexity and unpredictability of live industrial conditions are not fully captured. The reduced complexity may inadvertently overlook interactions and details that are critical to system reliability and security. Consequently, while laboratory simulations offer controlled insights into system behavior, they may not accurately reflect performance under the real-world operational stressors and dynamic factors inherent in a real-world environment. Therefore, special attention should be given to the issue of the performance and detection of bottlenecks that may arise in cybersecurity systems. That kind of test has been performed and teh results are presented in Section 5.

The design, implementation, and maintenance of a proper laboratory test setup is expensive. All significant changes in hardware and software implemented in the actual system must be considered, and the laboratory environment must be modified accordingly. In addition to cost, we have to rely on highly skilled professionals who can deal with the laboratory setup, which is also expensive. Finally, cyber threats are constantly modified, particularly in a way that is hard to detect and mitigate. Therefore, the testing methods used in the laboratory must also be updated; software and hardware infrastructure modifications are also necessary in some cases.

All things considered, it is important to have a laboratory test environment that mimics the behavior of the real critical infrastructure. It allows us to efficiently and inexpensively carry out many tests safely. Unfortunately, it is both a technological and financial challenge.

When limited resources, time, and money are available, a practical solution is to design and implement only a simplified replica of the real system in the laboratory, where we only consider the necessary infrastructure elements and aim to mimic only the key subsystems. As a result, it is possible to obtain a functioning laboratory infrastructure, but we have to remember the existing simplifications and constraints.

The laboratory test infrastructure used in this work is presented in Figure 3. We used a simplified laboratory infrastructure due to limited resources and time. The structure consists of a switch to which the server, test station, and programmable logic controller (PLC) are connected. The switch used has a switched port analyzer (SPAN) function, often called “port mirroring”, that allows for traffic from one or more ports on the switch to be copied onto another port, which can then be analyzed by a monitoring tool. SPAN is particularly useful for monitoring, diagnosing network problems, and analyzing security. The architecture presented allows for the traffic to be replayed on the server and PLC, as well as for communication generation between the test station on the server and PLC. Recreating any traffic and redirecting it to the SPAN port prior to probing analysis are possible.

The goal of the laboratory environment is to obtain the network traffic as close as possible to that which is observed in the real facility. Therefore, the focus is solely on network traffic, minimizing the contribution of specialized equipment. To achieve such a formulated goal, a laboratory infrastructure has been designed and implemented using specialized tools that allow for the mapping of any network traffic in a controlled manner. We assume that the generated traffic in the test network came from the real installation and, for the purposes of testing, was deliberately modified to simulate various network attacks.

4. Software Tools

To develop a laboratory test environment with reduced hardware requirements, we have to use software tools that allow for the network traffic to be recorded, modified, and replayed in a controlled manner. We can acquire the original communication, make targeted modifications with attack characteristics, and then play the traffic back. Hence, we can simulate a complex attack using a simple test infrastructure.

4.1. Recording the Network Traffic

From the point of view of testing a network’s security and performance, it is necessary to record the network traffic. In this work, we consider the following two software tools: Wireshark (ver. 4.0.1) and Tcpdump (ver. 4.99.5).

The free tool Wireshark is one of the best-known and widely-used tools for capturing and analyzing network traffic [56]. It is available for many operating systems, including Windows, Linux, and macOS. It provides in-depth packet analysis at the protocol level and offers a convenient graphical user interface. It supports capture and display filters, which allows one to easily search for packets of interest. TShark is a version of Wireshark that uses only command-line communication with the user. In contrast to Wireshark, more effort is required to configure and use this tool. Secondly, it is a rather resource-intensive program, which can cause problems when only low-performance or old machines are available. Therefore, for simple tasks on servers where no graphical user interface (GUI) is possible, tcpdump is recommended.

Tcpdump is a command-line-style tool for capturing packets on Unix/Linux systems [57]. It is also available for Windows, under the name WinDump (ver. 4.3). It is renowned for its low system resource requirements and high speed. As a result, it is a great solution when we use servers without a graphical user interface. It saves captured packets to .pcap files and allows for filtering during capture.

In this work, tcpdump is used to capture and log network traffic on Linux servers. Data collection is performed automatically. Next, Wireshark is used for further analysis, understanding the dependencies and selecting the special data fragments.

4.2. Modification of the Network Traffic

Modifying network traffic is the second step in designing tests in a laboratory infrastructure. There are many tools to perform this task [58]. The most popular ones used in the project are listed below. In addition to capturing and analyzing network traffic, Wireshark allows for filters to be set and modified data to be saved in this way. Unfortunately, Wireshark does not offer packet editing capabilities, so its use is limited for the currently considered purpose.

Scapy (ver. 2.5.0) is a very flexible tool for working with network packets [59]. It allows for the manipulation of virtually every aspect of a packet [60], e.g., creating new packets from scratch, modifying captured packets, or generating and sending them to the network. Additionally, Scapy allows us to have full control over packet design. It allows for packets to be created with any combination of layers, e.g., Ethernet, Internet Protocol (IP), Transmission Control Protocol (TCP), defining their values such as IP addresses, ports, or flags. Scapy can load existing packages from a .pcap file and modify them. We can then edit any field in the packet header. Scapy allows for the editing of transport layer data, such as TCP and User Datagram Protocol (UDP), and application layer data, including data sent over the Hypertext Transfer Protocol (HTTP) or other protocols. It allows for the editing of ports, flags, and specific fields in the packet payload. Scapy handles the dynamic modification of packet fields. It means that when one field is changed, other dependent values, such as the checksum, can be automatically recalculated, ensuring the packet’s integrity. Edit mode allows for changes to be made without creating the entire packet from scratch. Once modified, packets can be saved to a new .pcap file for later analysis or network restoration. Scapy also allows for packages to be exported to various formats. In this work project, the Scapy package is used to modify MODBUS protocol packages, function codes, IP addresses, and register values.

Ettercap (ver. 0.8.3) is mainly designed for man-in-the-middle (MITM) attacks that enable the real-time interception, analysis, and modification of network traffic [61]. It can handle both Address Resolution Protocol (ARP) and Domain Name System (DNS) spoofing attacks. Additionally, it can inject malicious code and modify packets on the fly. As a result, Ettercap is widely used in network security testing.

The most common MITM techniques in Ettercap ARP spoofing involve impersonating MAC addresses on the local area network (LAN). It allows for the interception and modification of traffic between two devices, e.g., a router and the victim’s computer. DNS spoofing involves altering the DNS response to redirect traffic from a target website to another, such as a fake phishing site. It is also possible to use Internet Control Message Protocol (ICMP) Redirect, which is the manipulation of ICMP messages to redirect traffic through the attacker’s computer.

Ettercap can analyze unencrypted protocols such as FTP and HTTP in real time. Thanks to such a feature, it is easy to capture login details and other data.

Ettercap allows for the creation of filters to manipulate packets in real time. Filters make it possible to alter the content of transmitted data, inject malicious code, or capture passwords and other sensitive information. They act as scripts that allow for the modification of selected fields in a packet. This functionality has been used to modify the contents of MODBUS TCP packets in attacks carried out using GAN [62,63,64]. The modified network traffic files have been corrected using the tcprewrite command of the Tcpreplay library to correct the checksums of the modified packets.

4.3. Replaying the Network Traffic

The final testing stage in the laboratory infrastructure is to recreate the previously modified network traffic and conduct tests for the test scenarios. The following two software tools have been used during our study: Tcpreplay (ver. 4.4.4) and Caldera (ver. 4.2.0).

Tcpreplay replays recorded network traffic from .pcap files. It is frequently used for performance evaluation, security analysis, and the testing of network applications and hardware [65]. Tcpreplay replays the network traffic exactly the way it was recorded. As a result, we can test the network by considering their response to specific traffic patterns.

Tcpreplay can replay packets to help test the performance and effectiveness of firewalls and IDS/IPS systems. As a result, we may replay traffic containing known attacks to see if the implemented detection algorithms work.

Tcpreplay can replay large .pcap files with a large number of packets in a short period of time. It allows us to evaluate the network’s reaction to high traffic and helps us to simulate denial-of-service (DoS) or DDoS attacks.

Tcpreplay offers the tcprewrite utility, which lets the user modify .pcap files before replaying them. Different settings can be modified, e.g., the IP and MAC addresses, source and destination ports, and other fields in packets, which is useful if the replayed network traffic is redirected to a specific machine or the device under test requires specific addresses.

Caldera was developed by MITRE (McLean, VA, USA) [66]. It is an open-source platform for simulating cyberattacks and automating security testing based on techniques listed in the ATT&CK framework [67]. It can perform simulations of realistic attack scenarios in a controlled environment. As a result, we can evaluate and assess the systems’ ability to detect and respond to threats. Although Caldera does not replay the recorded network traffic like Tcpreplay, it uses various offensive techniques to replicate adversary actions, e.g., advanced attacks. Caldera emulates the entire attack lifecycle and simulates techniques dynamically to reflect real-world threat scenarios.

5. Cyberattack Scenarios and the Obtained Results

Using the laboratory structure described in the previous chapter, tests were conducted on the developed detectors in a Linux server environment and in a PLC environment. The tests were carried out using various attack generation techniques. The MODBUS protocol used generative antagonistic networks (GANs) and Caldera software solutions to modify the register values. Caldera was also used for both the network scanning and man-in-the-middle attacks. In addition, performance tests were conducted.

The tests were focused on single attacks, which could be combined into a more sophisticated cyber kill chain scheme, often posing a long-term threat of the ATP (advanced persistent threat) type. It is worth noting, however, that the ability to replay individual elements and demonstrate that the security features are sensitive to this type of attack allows one to conclude that the tested protections will also be effective against complex attacks.

5.1. Attacks on Linux Probe Generated Using GAN

One of the ways to test the performance of the anomaly detection algorithms developed in the project (potentially caused by cyberattacks) is to use GANs. Conducting such tests using GANs would require direct interference with the mine’s network traffic to modify the transmitted process data. For obvious reasons, such an action would be impossible since the actual mine installation could not be the target of such a cyberattack. For this reason, we decided to carry out the attack using previously provided .pcap files, which contain a record of the actual network traffic from the mine. The implementation of the test consisted of modifying the .pcap file containing network traffic with the MODBUS TCP protocol, as well as with the process data in the selected communication channel (sender, receiver, unit ID, and set of registers) substituted with the values generated by the GAN generator(s), and then running an anomaly detector for the network traffic reconstructed from the modified .pcap file. The network traffic is then run on the server by sending it to the loopback. As a result, we can obtain conditions identical to those at the mine, where network traffic is redirected to a Linux server identical to the one used in the laboratory.

5.2. Modification of Network Traffic

To create modified .pcap files from the “swtx.pcapng” file with the original (unmodified) network traffic, 5 min of traffic were cut out, from 10 to 15 min. Then, the communication channel and data generation method (one GAN for all bytes of register values or one GAN for each byte) were selected. In the end, the .pcap file with 5 min traffic was modified as follows:

• from 0 to 2.5 min: traffic unchanged;
• from 0 to 2.5 min: the values of registers in the MODBUS TCP frames in the selected communication channel are replaced with values drawn from the previously created files.

In each case, the content of only one selected channel was changed.

The ettercap tool is used to modify .pcap files, allowing for the splitting of network traffic files into parts and the modification of the traffic contained within them using specially prepared filter files, among other things. In our case, the program generated these filters automatically at the beginning of the .pcap file modification process. In addition, the modified files were corrected using the tcprewrite command of the Tcpreplay library to correct the checksums of the modified packets.

5.3. Test Results of Cyberattack Generation Using GANs

For testing purposes, a dozen files were prepared, each containing network traffic that was modified using learned generator GAN structures. The variety of values of the data transmitted guided the choice of communication channels. For each selected channel, a version was prepared with the register values generated from a single GAN and a version where each byte came from a separate GAN. It is worth noting that GANs may not fully capture the nuances of real-world threat scenarios, which can lead to incorrect performance evaluation of detection algorithms. To eliminate this problem, analyzing the generated data and ensuring its physical correctness are necessary. The study verifies the scope of the data and ensures its limited variability.

For testing, anomaly detectors were trained using the original network traffic and then run for modified traffic reconstructed from the prepared .pcap files. The Tcpreplay tool was used to reconstruct the traffic from the .pcap files. The generated traffic was used to test the quality of the anomaly detectors.

5.4. Attacks from the Caldera System on the Linux Probe

The centerpiece of the Caldera system is the server, which was installed on one of the computers in the laboratory. Individual agents were deployed on different devices and communicated directly with the server, receiving commands for execution and transmitting their results. Management was carried out using the server’s remotely accessible web interface, allowing for the management of agents, their capabilities (i.e., the pool of available attacks and commands), facts (i.e., configuration data of attacks), and entire operations (i.e., sequences of executed actions).

5.5. Standard Attacks on IT Infrastructure

Caldera offers a wide pool of possible actions for standard IT networks. The main attack scenario from this pool used in the project was the scanning operation, a necessary step in infiltrating the attacked system. This was a key step within the classic kill chain model of attack [68]. In the context of cybersecurity, understanding the stages of the attack chain helps organizations identify vulnerabilities and implement preventive measures. Infiltration, as a necessary stage of an attack, allows for the early detection of an adversary’s incipient activities. An attack carried out as part of tests for the probe software running on a Linux server, as part of a cybersecurity system, was correctly spotted by the probe detectors. Port scanning traffic was classified as an anomaly and treated as the detection of an attack attempt.

5.6. Attacks on MODBUS Protocol

The Caldera system includes the Caldera-OT plug-in, which includes a set of capabilities that enable attacks on three industrial protocols (BACNet, DNP3, and MODBUS). The MODBUS plug-in equips the agents in the Caldera system with the ability to undertake read or write operations on the bits and registers of any device, a simple but also very likely attack case. A more interesting variant is to use the fuzzing implemented in this plug-in, i.e., the ability to automatically write random values to random registers of the indicated devices. This is an attack designed to destabilize the operation of controllers and, in extreme cases, achieve an abnormal state, leading to their unavailability.

5.7. Man-in-the-Middle Attack on Linux Probe

The project developed a tool to perform man-in-the-middle attacks on MODBUS connections that are in progress. The tool intercepts ongoing connections using the ARP spoofing technique and applies active filtering to them. This technique allows for interference with the connections themselves (e.g., discarding selected packets to prevent or obstruct communication) and with the data transmitted in them (in particular, the swapping of the values of individual registers returned by devices was tested).

A MODBUS protocol connection was set up to perform the communication interception attack detection test, running the Caldera software. The Caldera software intercepted communications by introducing a delay in transmission. The detector correctly recognized the resulting delay. The transition time between states increased beyond the tolerance threshold, which was classified as an anomaly and a potential attack.

5.8. Probe Performance Tests on Linux

In addition to the domain tests performed to check the detectors’ sensitivity, performance tests were performed. A dedicated client–server connection was set up over the MODBUS protocol to perform a performance analysis of the detectors running on the probe Linux environment. The MODBUS client was run on the PLC side, while the MODBUS server was run on the Linux server side. Traffic was routed to the configured SPAN port from which the probe detector received data. The client cyclically executed function 16 (writing multiple registers), and tests were performed for the different query frequencies (defined by sleep time) and the number of registers written.

Figure 4 shows the change in the transfer amount between client and server depending on the number of registers. As expected, the traffic to the server increases as the number of registers written increases. We note that the increase is linear and proportional to the number of registers. Traffic from the server does not change, as the server only confirms the execution of the registers written.

Similarly, the CPU load changes linearly as the number of registers increases, as shown in Figure 5.

Figure 6 shows the change in transfer size between the client and server depending on the writing frequency. The frequency is determined by the sleep time. Reducing the sleep time doubles the traffic twice, resulting in an exponential relationship.

Similarly, the CPU load changes exponentially as the sleep time decreases, as shown in Figure 7.

The goal of the tests is to determine the limiting query frequency and the number of registers checked. In short, the limit values, at 95% of CPU consumption, are obtained with 100 registers written at a frequency of 70 Hz. The collected data show that the probe detector has limitations; however, for the monitored traffic (from the selected zone), they do not cause any problems. The amount of monitored traffic from the selected mine zone does not exceed the limits.

During the tests, RAM consumption was additionally monitored. It was noted that increasing the amount of network traffic had no significant effect on the probe’s RAM consumption. RAM consumption depended on the number of communication channels in the observed traffic. In the monitored run, the number of channels was constant.

The collected results allow us to assess the dependence of resources as a function of the number of registers and scanning frequency. The analysis conducted in this area provides valuable insight into the scalability and robustness of the proposed security solutions. It is worth emphasizing that the evaluation of resource consumption during intensive use can reveal bottlenecks and inform necessary optimizations, which are key to developing a resilient and efficient network defense system.

5.9. Probe Performance Tests on PLC

The software installed on the PLC in a Windows environment was identical to that of the Linux server. Therefore, repeating the quality tests was not necessary. It was much more important to check the performance of the PLC, which has significantly fewer resources than the server. Therefore, tests within the controller were limited to performance tests only. Due to the PLC’s single interface, the tests were carried out using the Python library SciyPy (ver. 1.10.1). As with the server tests, the tests were based on modified baseline traffic. The difference was in the way the traffic was reconstructed. In the case of the PLC, the scapy functionality was used, and the reading traffic from the files was routed directly to the probe’s programming input. The monitoring of 132 signals was run on the controller, and each signal was validated using five types of detectors. After running the complete solution, the memory usage was at 41%, and the CPU usage was at a high level of 90% (that is, high), but the controller worked stably and was responsive. The tests performed confirmed the performance of the solution implemented on the controller.

6. Summary and Conclusions

The properties of the tools discussed above were used in a project to develop detectors for network traffic anomalies at a coal mine. Due to the inability to perform quality tests in the mine facility, a test laboratory environment was developed. It was intentionally simplified compared to the industrial infrastructure but allowed for the reproduction of traffic prevailing in real industrial conditions. The developed test environment and the tools used made it possible to reproduce real conditions in a very short time with the involvement of minimal financial resources. Furthermore, the tests helped us to verify the effectiveness of the developed cybersecurity algorithms. In addition, the operation of the probe on a PLC and a Linux server was confirmed.

Using the developed laboratory environment and the software tools, tests of the detectors were successfully carried out in the Linux server environment and the PLC environment. Various attack generation techniques were used during the tests. GAN neural networks and Caldera software solutions were used to modify the register values in the MODBUS protocol. Caldera was also used for both the network scanning and man-in-the-middle attacks. In addition, performance tests were conducted.

Importantly, the described virtual environment enabled advanced testing in a piecemeal fashion, limited to the scope of testing, thus greatly minimizing the need to purchase dedicated hardware. The ability to simulate traffic in virtual environments, such as containers or virtual machines, eliminates the need to invest in physical testing hardware, further increasing testing flexibility. The developed solution was successfully used to verify security solutions for the Modbus protocol.

This article presents the experience gained while developing a cyber security system for a mine. Special attention was paid to the limitations and conditions occurring in the real industrial installations of such critical infrastructure. In particular, the absolute prohibition of disrupting communications resulted in the inability to conduct tests in a production environment. The solution to such constraints was to develop a methodology to recreate a realistic test environment with minimal time and money. The developed approach focused on security, cyberattacks, and vulnerabilities, describing the input data and the tools used to carry out the attacks. During the preparation of the data, attention was paid to the issues of data selection and the aspect of performance as key features to achieve reliable results to demonstrate the effectiveness and robustness of the proposed system in dealing with cyberattacks in a real coal mine environment.

Further work has been planned to consider other communication protocols and to evaluate the effectiveness of other anomaly detection algorithms. Secondly, we intend to apply the presented approach to other processes. This is because the presented approach to replicating the IT/OT infrastructure is flexible and can be applied not only in mine systems but also in different industrial systems. Moreover, it can find its use in embedded systems, e.g., automotive applications [69], robotics [70], and drones [71].