Cyber–Physical Systems: The Last Defense

Abstract

The development, evolution, and operation of a cyber–physical system are cross-domain, holistic processes. The process encompasses all elements of a cyber–physical system, including computation infrastructure, software, interfaces to the physical world, human interactions, and safety and security engineering. The process is holistic because it must assure conceptual integrity and correct interoperability across all elements of the CPS. Unfortunately, at every stage of this process, vulnerabilities can be introduced into the system (due to negligence, mistakes, lack of skills, malicious activities, etc.). These dormant vulnerabilities can cause failures of the runtime system, possibly resulting in damage, loss of property or life, safety accidents, or security incidents. A promising approach to mitigate such risks is runtime anomaly detection using artificial intelligence/machine learning. This tutorial paper introduces the fundamental concepts of AI/ML anomaly detection and describes the corresponding intervention mechanisms. Automated intervention mechanisms are the last line of defense against failures, faults, malfunctions, and malicious activities—and their unfortunate consequences. The paper remains at the conceptual level and defers implementation details to subsequent publications. The content addresses advanced students (at the master’s level) and researchers entering this fascinating field.

1. Introduction and Context

Cyber–physical systems [1,2] are a double-edged sword. On the one hand, they are extremely useful and enable numerous indispensable applications. On the other hand, they pose a considerable risk and may lead to safety accidents or security incidents. Therefore, safety and security are the critical quality properties of a trustworthy cyber–physical system. Unfortunately, every day, safety accidents and security incidents—some with grave damages—are reported. Safety accidents and security incidents are always the consequence of a vulnerability in the cyber–physical system.

Cyber–physical systems (CPS) result from a process encompassing their development, evolution, and operation. The process is cross-domain because it encompasses all elements of the cyber–physical system, including computation infrastructure, software, interfaces to the physical world, and human interactions, as well as safety and security engineering domains. The process is holistic because it must assure complete conceptual integrity and correct interoperability across all elements of the CPS.

Figure 1 shows the threat context of a cyber–physical system. Vulnerabilities can be introduced during the development process or appear in the elements of the runtime system. Sources of vulnerabilities during the development process include weaknesses in the development process, deficiencies in development tools, vulnerabilities imported through third-party software, deficits in algorithms or training data, and, last but not least, insufficient risk analysis and risk management [3]. Vulnerabilities introduced during development are transferred to the runtime system! During operation, additional vulnerabilities appear, such as emergent behavior and emergent properties [4,5], autonomous decisions, malicious attacks, errors or faults in an element of the runtime system, failures in the execution platform, systems software weaknesses (Operating system, database, browser, etc.), sensor and actuator malfunctions, and uncertain or unpredicted operating conditions.

In many cases, the cause of a safety accident or security incident is software-related [6,7]. Fortunately, a vast and valuable literature exists for building safe and secure systems [8,9,10,11]. However, the undeniable fact remains that cyber–physical runtime systems contain hidden vulnerabilities that may lead to safety accidents and security incidents.

In addition, today’s software development suffers from three challenges—complexity, change, and uncertainty—known as the “Three devils of systems engineering” [12]. The three devils often force damaging compromises on design and implementation decisions!

2. Impossibility Statement

The storyline of safety and security in cyber–physical systems begins with the “Impossibility Statement” (Figure 2, adapted from [13]). The consequence of the impossibility statement is that the deployed runtime cyber–physical system contains unmitigated vulnerabilities that may lead to safety accidents or security incidents. The last defense against such hidden vulnerabilities is runtime monitoring with automatic, preferably real-time intervention. A powerful technique for this task is anomaly detection, particularly when combined with artificial intelligence/machine learning. An anomaly detected by a machine learning algorithm is an observation, event, or pattern in the real data that deviates significantly from the learned notion of normal behavior and is therefore flagged as unusual or unexpected relative to the majority of the data, thus requiring intervention (https://en.wikipedia.org/wiki/Anomaly_detection, accessed 10 November 2025). The real-time intervention mechanism encompasses all the elements for runtime protection of the CPS and thus constitutes the last line of defense before a safety accident or security incident occurs.

3. Cyber–Physical Systems Behavior

The cyber–physical runtime system has two categories of behavior (Figure 3):

(1)

Desired behavior: Safe and secure operation, complete adherence to specifications, full conformance with laws and regulations, respecting all timing conditions, satisfactory handling of all errors, faults, and failures, comprehensive interaction with the user, and trustworthy in all situations and operating conditions;

(2)

Undesired behavior: Unsafe, insecure, dangerous, incomprehensible, untrustworthy, or erratic behavior.

Once every reasonable effort has been made during development and evolution to detect, identify, analyze, and eliminate as many vulnerabilities as possible, the runtime system must be protected against undesired behavior.

4. Cyber–Physical Systems Runtime Monitoring

Runtime monitoring is not a new technology. It has, for example, been introduced as Onboard Diagnostics (OBD) as an integral part of vehicle design for decades, and OBD-II became mandatory in the USA in 1996 [14]. An extensive literature on runtime monitoring and its applications exists [15,16,17,18,19,20].

The key idea of runtime monitoring is shown in Figure 4: The behavior of the runtime system is supervised, and additional hardware and software attempt to detect and identify undesired (anomalous) behavior. If such behavior is detected, the analysis starts, and a suitable response is launched.

“Classical” runtime monitoring is based on preprogrammed algorithms; i.e., when faulty conditions are detected, a preprogrammed intervention is triggered [15]. This type of runtime monitoring is static. Modern runtime monitoring utilizes machine learning and is expected to detect and identify many more, as well as unforeseen, abnormalities; thus, this type of runtime monitoring is dynamic.

5. Monitor–Analyze–Intervene

Figure 5 introduces the four elements of a machine learning last defense architecture:

(1)

The deployed cyber–physical runtime system in its operating environment;

(2)

The anomaly detection (symbolizing the necessary hardware and software for the application at hand);

(3)

The intervention planning element;

(4)

The intervention execution blocks.

5.1. Monitor

The anomaly detection element [21,22,23] consists of two parts: a static and a dynamic part. The static part consists of programmed algorithms. The inputs from the runtime system are observables (e.g., sensor and actuator values, and internal variables such as speed or position). As reference inputs for the desired behavior, the static part uses functional specifications (including quality property requirements), a set of rules (e.g., bracket values for output signals), policies, models, log files, and operational data [15]. Anomalies are detected if discrepancies between the inputs and the reference information (model) are discovered.

The dynamic part contains AI/Machine learning. Its inputs are information streams [24] from the cyber–physical runtime system, such as video, LIDAR, Radar, trajectory information, and sensor data value sequences. Of most significant importance for the dynamic part is the training data [25]. Two sources of training data are shown in Figure 4:

• Data obtained from the real world;
• Synthetic data: data generated by algorithms, artificial intelligence, or simulations [26,27].

In many modern applications, there is a lack of real-world data, particularly for critical scenarios like serious safety accidents. In such cases, simulations provide precious and broad training data, such as car accidents and their successful evasive maneuvers. Interestingly, video game technology provides effective support for generating synthetic training data [28].

5.2. Analyze

During the analysis phase, the ML algorithm compares the actual behavioral data of the cyber–physical system with a previously learned “normal” behavior [29]. A conceptual diagram is shown in Figure 6. The first engineering step is to select the data and context to be used in the actual application. The second step is to construct a machine learning model for normal behavior, trained on real or simulated data, or both. As a third step, anomaly criteria must be defined as rules or parameters for the model. The system is now ready for operation and can be deployed.

In many cases, incoming data streams are subjected to a noise reduction or removal algorithm [30,31]. The ML algorithm then analyzes the data stream(s) and detects irregularities. Lastly, the decision gate classifies into <normal>, <outlier>, or <anomaly>. Note that outliers and anomalies are not the same: An outlier is a statistically rare but possible point. An anomaly is a value that is not supposed to exist [29]. Once an anomaly is identified, an intervention is triggered. Many systems continue to use machine learning during operation to enhance their detection capabilities.

5.3. Intervene

The intervention phase is the most risky phase of the last defense: because autonomous machine decisions directly impact the physical world, they can cause or even enable safety accidents or security incidents. Figure 7 shows the autonomous intervention concept. Once it is determined that a situation is anomalous, the algorithm must decide which action to take. The actions can be preprogrammed, such as limiting the range of actuator outputs or monitoring the rate of change in sensor inputs. The more advanced forms are learned actions, such as ship collision avoidance.

The action decision algorithm can (Figure 6):

• Directly influence the behavior of the control system, i.e., block, change, or override functionality;
• Override or correct sensor or actuator values, i.e., protect the system from harmful input or output values;
• Manipulate the operating environment of the CPS.

Examples of actions are:

• Isolating a network node immediately after an unknown thread in the operating system is detected;
• Shutting down a TCP/IP port when excessive in- or outbound traffic is measured;
• Taking over a ship’s control when a threatening collision course is computed;
• Setting all traffic lights by the involved car to red after an accident at the intersection;
• Automatically take evasive action when a car recognizes an imminent hit of a pedestrian, animal, or cyclist;
• Force the CPS into a safe state, e.g., trigger an emergency stop of the train;
• Switch to a safe, degraded mode of operation, e.g., limit the maximum speed of a car to 80 km/h after a failure of the antiskid control;
• Close the system access for an employee after unauthorized data access attempts;
• Contact a cardholder after untypical payment behavior is detected from their card.

A possible autonomous intervention mechanism is shown in Figure 8. The training part is as described above, usually using real, simulated, or hybrid data. Here, multiple anomaly detection algorithms are used in specific applications, and a voting system decides, thereby reducing false positives and false negatives [32].

Unfortunately, today’s most significant obstacle to automatic, real-time intervention is the lack of binding, comprehensive, actionable laws, regulations, and liability provisions [33].

6. Machine Learning Algorithms

Various machine learning algorithms exist, and new ones are continuously under development [34]. In this tutorial paper, two algorithms are used:

• Deep reinforcement learning [35] for the AI safety example. Reinforcement learning is a machine learning paradigm in which the machine learns by interacting with the context/environment and collects rewards or penalties according to a gain function, to maximize cumulative reward over time;
• Unsupervised learning [36] for the AI security example. Unsupervised machine learning is a machine learning paradigm in which algorithms learn patterns and structure directly from unlabeled data, without using explicit target outputs or example answers to guide learning.

7. Safety: Last Defense

AI/ML for runtime safety in CPSs is a relatively new, exciting engineering discipline. Therefore, no consolidated body of knowledge or accepted development methodology is yet available. Due to the limited space in this publication, an illustrative example is used to demonstrate the power of AI/ML in safety. Note that this is not an experiment, but a conceptual sample demonstrating the power of AI/ML for anomaly detection. The example chosen is autonomous collision avoidance for ships [37,38]. Ocean-going ships face ever-increasing traffic, larger and faster vessels, and a shortage of experienced mariners. Therefore, ship automation, up to fully autonomous ships, is important for the future of sea traffic.

A significant step towards autonomous sailing is fully autonomous collision avoidance. Autonomous collision avoidance for a ship either supports or overrides (⟹Protective shell) the mariners or guides the ship autonomously.

This example uses constrained Deep Reinforcement Learning as AI technology (RL) [32,39]. Deep reinforcement learning AI is distinguished from supervised or unsupervised learning because it aims at long-term goals or policies [40]. The reinforcement learning AI determines the collision-free trajectory. However, the trajectory is constrained by the IMO’s (International Maritime Organization) “COLREG”—the mandatory set of rules preventing collisions at sea [41,42].

The example setup is shown in Figure 9. The ship (“Own Ship”) is enclosed in two “bumper zones”: An inner bumper for congested waters and an outer zone for open seas. If an obstacle, such as another vessel, a buoy, or a stationary obstacle, enters the bumper zone, a negative reward for intrusion is assigned to the RL. This value function guides the learning process.

Due to the scarcity of real-world training data, the training process utilizes simulation. This is achieved by providing a large number of simulated traffic scenarios during the training phase. After the successful training, the trained neural network is transferred to the operational RL. Note that the training continues during actual sailing, continuously and consistently improving collision-avoidance capability.

8. Security: Last Defense

AI/ML for runtime security in CPSs is a relatively new, promising engineering discipline. Therefore, no consolidated body of knowledge or accepted development methodology is yet available. Because of the limited space of this publication, again (as in safety above), an illustrative example is used to demonstrate the power of AI/ML in security. Note that this is not an experiment, but a conceptual sample demonstrating the power of AI/ML detection. The example is unsupervised learning for intrusion detection on the MIL-STD-1553 communication bus [43,44].

The MIL-STD-1553 communication bus (https://www.milstd1553.com/, accessed on 10 November 2025) is a message-based local communication system widely accepted for over five decades. Many legacy and modern systems (e.g., the F-35) are based on the MIL-STD-1553. Unfortunately, research has uncovered a number of vulnerabilities that may allow a range of attacks. The MIL-STD-1553 standard was introduced at a time (1973) when cybersecurity was no primary concern. Because today MIL-STD-1553 is a critical component of countless military and civilian platforms, a protocol change is not an option. Therefore, a robust defense in the form of intrusion detection is required!

Figure 10 shows the architecture of the example: it represents an intrusion detection system based on unsupervised machine learning that uses anomalies in bus traffic. The MIL-STD-1553 follows a strict, deterministic message exchange protocol. This time-based regularity yields a set of relevant features that define the model for regular (“normal”) operation.

The example has three components (Figure 9):

(1)

The MIL-STD-1553 bus with the bus controller, the bus monitor, and 1…31 bus participants (= remote terminals);

(2)

The AI/ML learning component. The first function is a traffic-generation simulator that generates malicious traffic patterns based on a list of attack vectors. The second function is the feature extraction parameters based on a predefined feature list. The relevant feature parameters and their anomaly decision parameters, together with the decision criteria (thresholds), are then provided to the anomaly detection mechanism;

(3)

The anomaly detection mechanism. This mechanism monitors bus traffic and detects anomalies using the CUSUM (Cumulative Sum) algorithm [45,46]. This statistical change-detection algorithm [47] identifies when a time-series feature deviates from a reference one. Finally, the decision mechanism—usually based on thresholds—raises an alarm if an anomaly is suspected and triggers defense mechanisms, such as isolating suspicious nodes.

9. Protective Shell

The idea of protecting a cyber–physical system against the misbehavior of the artificial/machine learning algorithm during runtime was presumably first presented by Lance Eliot [48]. He proposed it under the name of «AI Guardian Angel Bot» to improve the trustworthiness of machine learning systems. Here, the more technically oriented (and less esoteric) term «Protective Shell» is used.

Figure 11 shows the protective shell [48,49]. During development, evolution, and deployment of the system, all reasonable care is applied to detect, identify, analyze, and mitigate as many vulnerabilities as possible. However, some vulnerabilities inevitably find their way into the runtime system. To mitigate these hidden vulnerabilities, the protective shell includes additional hardware and software to detect, analyze, and respond to anomalies, thereby protecting the system from safety accidents and security incidents during runtime.

In modern CPS systems engineering, the need to supervise an AI/ML-based mission-critical cyber–physical system at runtime has become an accepted best practice [50,51,52]. Protective shells are an active field of research. Figure 12 presents a conceptual architecture for a protective shell: it is based on the well-known and proven MAPE-K concept (Monitor-Analyze-Plan-Execute with Knowledge [53,54].

The monitoring part reads, preprocesses, and fuses all the available information from the runtime system. The required information is defined during the development process in the safety assurance and security assessment cycles.

The analyze part examines monitoring information and detects, identifies, and classifies anomalies using both preprogrammed scenarios and AI/ML-trained models.

The plan module searches for an appropriate response to protect the application, preferably in real time. Here, both preprogrammed and learned information are used.

The execute module transforms the reaction decided by the plan module into an actual intervention to the runtime system. Note that possible intervention mechanisms must also be defined and implemented during the development cycle. This requires special attention and care.

10. Extended Attack Surface, XAI, and Robust AI

Implementing AI/ML-based runtime monitoring (i.e., a protective shell) introduces four new risks:

(1)

The protective shell increases the system’s complexity by introducing additional hardware, software, and processes. As experience shows, additional complexity also enlarges the attack surface [55], i.e., generates new failure modes [56] and enables new attack vectors [57]. Unfortunately, effective kill chains, i.e., specific attack methods and tools, are readily available to target AI systems [58]. A meticulous and thorough risk analysis for the protective shell is therefore required.

(2)

Artificial intelligence and machine learning may execute autonomous, inexplicable (although possibly correct) decisions and actions. This autonomous behavior may cause legal, audit, and regulatory issues, but it also poses the danger of harmful decisions. Therefore, safety- or security-critical AI applications must rely solely on explainable artificial intelligence (XAI) mechanisms [59,60] and safety-certified algorithms [52,61].

(3)

In AI-based safety- or security-critical CPS, severe damage can result from their malfunction. Failures, malicious activities, deficient learning, and other issues can cause such malfunctions. To guard against these, the AI must be safe and robust. Fortunately, the field of safe and robust AI is developing rapidly, and sources are available [60,62,63,64].

(4)

Unethical use of AI/ML: AI/ML may be used in applications that ignore or violate ethical standards, thereby causing incidental or accidental harm to people, organizations, social structures, or the environment [65,66].

11. AI/ML Laws and Regulations

As with any technology, AI/ML must be embedded in the legal and sociological systems of Nations. As a first step, regulations must be proposed and then transformed into local or international law. Many efforts are underway or completed, e.g., for AI/ML regulations [67,68,69]. Unfortunately, the individual regulations in different Nations seem to diverge strongly.

Also, great efforts are made in most Nations to develop laws governing the development, governance, use, and evolution of AI/ML [70,71]. Complying with existing law and adapting to evolving law is becoming a significant challenge and effort for businesses using AI/ML, and it also poses a significant commercial risk.

12. Forensics

The protective shell’s task is not complete while ensuring the safe and secure operation of the CPS. It must also protect the CPS against unjustified legal action and liability claims arising from a safety accident or security incident. The protective shell—in addition to the already installed data logging activities of the CPS—must collect, store, and protect sufficient information [72,73,74,75] to:

(1)

Defend against post-accident and post-incident unwarranted accusations;

(2)

Allow for authoritative discovery of the causes/sources of accidents and incidents, and use them to improve the CPS.

Note that in many cases, the information gathered during operations must be secured during transfer to storage to prevent accidental or malicious modifications. Here, a digital signature may be required.

13. The Value of Machine Learning for Safety and Security

Figure 13 shows a classification of the paths from the cause of a vulnerability to its exploitation and, finally, to the resulting safety accident or security incident. The cause may either be known (identified) or unknown (hidden). Two types of propagation paths from the cause to the resulting vulnerability of the system exist: deterministic (fully understood propagation path) or statistical (uncertainty in mechanism, timing, and effect). If the vulnerability is exploited, again, two propagation paths to the safety accident or security incident are possible: deterministic and statistical.

For each system, process, or product, the state-of-the-art provides a number of protection mechanisms, such as:

❶ = Generic protection mechanisms (technologies and methods applicable to all systems, processes, and products, such as a formalized, strict, and safety- or security-aware development process [10,76];

❷ = Best practices (technologies and methods applicable to specific systems, processes, and products, such as a firewall to protect the perimeter of an Internet-banking system [77,78];

❸ = Risk Assessment and Management Process [79] aiming to identify and mitigate as many risks as possible;

❹ = Machine Learning [80,81].

Figure 13 shows the relative importance of the four protection mechanisms for the eight cases, represented by the diameter of the circled numbers (size = relative importance for the case in question). The more uncertainty in the system exists, the higher the protection value of AI/ML becomes. Because modern systems, especially systems-of-systems, have tremendous complexity, they contain a mix of all the cases in Figure 13 and include all the causes of the “impossibility statement” (Figure 2)! Therefore, the adequate use of AI/ML is indispensable.

14. Open Challenges

Large-scale application of artificial intelligence/machine learning to the safety and security of cyber–physical systems is a relatively young research discipline. However, it has roots deep in computing history [82]. Therefore, open challenges exist, such as:

• Authoritative and law-conforming policies covering the application of artificial intelligence/machine learning for safety and security in different fields of application and diverse organizations;
• Dependable, transparent machine learning algorithms (Explainable AI) [83];
• Reliable, consistent, and fair training data (either from the real world or simulated), including their provisioning methods;
• Correct, timely, and effective intervention mechanisms, including their planning and decision mechanisms;
• Collection of sufficient, meaningful, and legally acceptable forensic data [73];
• Metrics for the evaluation of the efficiency and effectiveness of AI/ML usage;
• Defense against attacks on the AI/ML algorithms, their implementations, and training data;
• Development and enforcement of ethical principles, guidelines, and standards for AI/ML applications.

15. The Last Question: AI Ethics

This paper is about machines making autonomous decisions that impact the physical world. Erroneous, incomplete, late, or dangerous decisions can cause severe damage to people, organizations, property, or the environment. This fact raises the last question: How are governance, responsibility, liability, and ethical issues handled in autonomous AI/ML decision systems?

An interesting variety of emerging literature on AI ethics exists [84,85,86,87,88]. Ethical challenges will significantly influence AI/ML research and applications in the future.

Fortunately, valuable guidelines for AI/ML systems that respect ethical principles are available, foremost from the United Nations Educational, Scientific, and Cultural Organization [89]. This work defines four values, 10 core principles, and 11 actionable policies for the design, implementation, use, and evolution of AI/ML systems.

At the heart of AI-ethical organizations, services, processes, and products are an adequate ethics policy and assertive governance! A common misunderstanding is that an organization needs only one ethics policy to cover all activities. Unfortunately, ethical issues depend strongly on the application, e.g., fairness in an AI application evaluating job candidates or provable safety in an autonomous vehicle. Therefore, ethics is more of an organizational issue than a technical issue. As a consequence, each organization should implement an ethics framework [69,90,91]. Figure 14 shows the main elements of such a framework:

• A complete, actionable, enforceable ethics policy, aligned to laws, regulations, and general principles, and focused on the specific application;
• A strong, dedicated, and responsible governance (governance board, assigned responsibilities;
• Monitoring and auditing facilities (partially automated) to detect and correct ethics violations.

16. Conclusions

Many of today’s cyber–physical systems control mission-critical applications. Because they can directly influence the physical environment, they can enable or even cause safety accidents or security incidents. Best practices increasingly demand reducing these risks through runtime monitoring and real-time intervention to prevent or mitigate safety accidents or security incidents.

One promising mechanism for detecting imminent misbehavior and mitigating potential consequences is anomaly detection. The runtime system’s operation is monitored, and anomalous conditions are identified. To enable timely detection of anomalous behavior, not only preprogrammed scenarios are used, but also artificial intelligence/machine learning is applied.

The methods and mechanisms for anomaly detection, classification, and mitigation are the elements of the protective shell. The protective shell includes specific hardware, software, sensors, and processes for preventing and mitigating runtime system safety accidents or security incidents through anomaly-based detection.

Adding a protective shell to the runtime system increases the system’s complexity. The protective shell must therefore be developed with utmost care with respect to safety and security, including emerging behaviors.

Anomaly detection and mitigation of the possible consequences with AI/ML is a fascinating, active field of research. This tutorial paper presents an introduction to the relevant concepts.