Evolutionary Game for Confidentiality in IoT-Enabled Smart Grids

Abstract

We applied evolutionary game theory to extend a resource constrained security game model for confidentiality attacks and defenses in an Advanced Metering Infrastructure (AMI), which is a component of IoT-enabled Smart Grids. The AMI is modeled as a tree structure where each node aggregates the information of its children before encrypting it and passing it on to its parent. As a part of the model, we developed a discretization scheme for solving the replicator equations. The aim of this work was to explore the space of possible behaviors of attackers and to develop a framework where the AMI nodes adaptively select the most profitable strategies. Using this model, we simulated the evolution of a population of attackers and defenders on various cases resembling the real life implementation of AMI. We discuss in depth how to enhance security in AMI using evolutionary game theory either by a priori analysis or as a tool to run dynamic and adaptive infrastructure defense.

1. Introduction

Smart grid systems are intended to optimize the usage of electrical resources. These systems are complex Cyber-Physical Systems (CPS) exposed to various security challenges. An advanced metering infrastructure (AMI) is one of the main communication components of the smart grid that consists of communication networks connecting smart meters and collectors. The AMI collects and processes data from large number of devices and reports the results over communication networks. It can provide advanced services, such as monitoring, alarm, billing, remote home control, intrusion detection, fault tolerance, and software updates [1]. Monitoring the smart grid can contribute to grid stability. The alarm functionality can address alarms for components failures in the grid and/or alarms in the Smart Home. The billing functionality could be for total consumption every hour or max usage. The remote home control functionality can control home devices by interacting with, e.g., the heating system. The intrusion detection functionality can monitor hacking attempts to the home, the control center, and any entity in between.

While providing advanced services, AMIs also introduce several security risks including attacks on confidentiality. The report [2] shows that connected Internet of Things (IoT)/CPS devices will grow at a Compound Annual Growth Rate (CAGR) of 13 percent with 26.9 billion related to the IoT in 2026. The rise of large scale interconnectedness and often outdated design of the devices present a significantly expanded attack surface. Despite the significant efforts in securing IoT/CPS-enabled smart grid systems, many remain vulnerable to various advanced and evolving cyber-attacks [3,4,5]. There are several reasons for this. IoT/CPS-enabled smart grid systems rely upon wireless networks. It makes them vulnerable for eavesdropping. These devices may also be unattended for prolonged periods of time leaving them vulnerable to physical attacks, and most devices limited in energy and computing power do not allow for the implementation and use of complex security schemes [6,7]. A recent survey [8] has shown that False Data Injection (FDI) attacks also threaten state estimation in smart grids.

Furthermore, adaptive attackers will adapt their strategies to the current security situation, and to newly deployed countermeasures. Such emerging attacks can become very sophisticated and can be coordinated [9], persistent [10], and collaborative or cooperative with specialized attack expertise [11]. Examples of such coordinated attacks include data injection attacks concurrently occurring from multiple adversaries [12], large-scale stealthy scans, stuxnet worm outbreaks and Distributed Denial-of-Service (DDoS). Such adaptive, collaborative or coordinated attacks require adaptive, collaborative or coordinated defenses. Therefore, the confidentiality of AMI data must be protected within the AMI system, in transit or at rest, which requires significant collaboration, evolution and adaptation in the security of the AMI. Evolutionary game theory (EGT) lends itself to model the dynamic interplay between the way attackers adapt and evolve their behaviors as evolutionary attacks and the way defenders anticipate the unknown and prevent dynamic, adaptive attacks that evolve over time. Evolutionary game theory studies adaptive rules that govern dynamic behavior. It offers, a solid basis for realistic and intelligent decision making in an uncertain world, describing how individuals make decisions and interact in complex environments in the real world [13].

The components in the AMI need to collaborate to achieve a common goal in collecting, aggregating, transmitting and securing the data. The data confidentiality attack in the AMI is unauthorized access to sensitive information between utilities and users by targeting the AMI components, such as smart meters, data concentrators, communication networks, and a central, or head-end, system. Users’ consumption habits and other relevant information must be protected from access by unauthorized persons or companies. The data sets in the head-end system must only be accessed by authorized systems or users. Confidentiality is, therefore, one of the primary concerns in AMI. As demonstrated in Reference [14], integrity and confidentiality attacks cause monetary effects on the AMI which in turn have cascading effects to other interdependent critical infrastructures, such as health, finance, and telecoms. In this paper, we focus on evolutionary game for confidentiality attacks and defenses for the AMI.

Previously, we have introduced an evolutionary game framework [15] that models evolving attacks and defenses in connection with data integrity for smart grid systems. The novel contributions of this paper are:

• the formulation of the AMI evolutionary game and the derivation of a numerical scheme for this game,
• the simulations of the evolutionary game on realistic AMI cases for confidentiality,
• the identification of constraints, and
• analysis of the confidentiality evolutionary game allowing the defender to explore the space of strategies and to select the optimal set of solutions.

The remainder of the paper is organized as follows. Section 2 and Section 3 give a brief literature survey and present theoretical background for this work, respectively. In Section 4, we introduce our system and game model, formulate the confidentiality game as an evolutionary game using replication dynamics, and discretize the evolutionary game model and derive a numerical scheme for solving the evolutionary dynamics. In Section 5, we carry out simulations on relevant AMI cases for confidentiality and demonstrate how this can be used to inform the AMI security. The simulation results are presented and discussed in Section 6. Finally, we conclude and discuss the future work in Section 7.

2. Related Work

The IoT/CPS system brings great benefits to the cyber physical IoT-enabled smart grids by connecting people, processes, services, devices, and data. However, the rise of large scale interconnectedness presents a significantly expanded attack surface. There exist significant efforts to secure the IoT-enabled AMI, which is the core component of the smart grid. In this section, we give a brief review of these efforts. The research that studies game theoretical approaches for modeling the evolving nature of cyber-attacks inside the IoT-enabled AMIs is not addressed sufficiently in the literature. In this section, we present the work on modeling security threats for IoT-enabled smart grids with constrained processing resources. In this work, we use evolutionary game theory. Therefore, this section also discusses literature review regarding applications of evolutionary game theory for IoT-enabled smart grid.

The IoT-enabled AMI is an integrated system of smart meters, collectors, communications networks, and data management systems which support the safe, efficient, and reliable distribution of electricity and advanced functionality to energy customers [16]. Unfortunately, the power grids have been the target of sophisticated cyber-attacks which could lead to grid shutdown, cascading failures, damage to the infrastructure, and potential harm to people [17]. Such targeted attacks could have devastating effects on government, trade, commerce, banking, transportation, and other important aspects, which rely on energy to operate. A compromise of AMI may also result in an invasion of privacy and provide a surface from which to extract information from users, such as Internet activity, financial, or health records [17]. The AMI poses several well known security threats [18,19,20,21].

As the IoT-enabled AMI is the core component of the smart grid, it is thus important to identify the attack surface and protect it from cyber-attacks. The cyber-attack surface of the AMI has been quantified and examined [17]. It is also important to measure the significance of threats and how they can transpire into attacks in the AMI environment. Different categories of attack types and analysis of the various countermeasures against these attacks have been studied [22]. A methodology for assessing security, privacy and dependability in a combined manner in the smart grid has been developed and measureable security in smart grids has been introduced [1]. A controlled Markov-Gaussian process has been suggested to minimize the damage of advanced persistent threats in cyber-physical systems [23]. He and Yan [24] provide a systematic review of the critical cyber-physical attack threats and defense strategies in the smart grid, as well as discuss a wide range of opportunities and challenges in enhancing energy security by maintaining the integrity of smart grid under complex cyber-physical attacks. Ismail et al. [25] presented a noncooperative game for attacks on data confidentiality for smart-grid AMI and studied the strategies of the attacker and the defender at the Nash equilibrium. Applying this model, the authors defined the optimal strategy of the defender and the minimum resources required for defending the assets.

Evolutionary game theory is a branch of game theory. Evolutionary game theory, rooted in classical game theory and the theory of evolution [26], has been effectively studied to model population dynamics in biology and economics domains, but its application to smart grid security has not been fully exploited. Santos et al. [27] argue that, by using a dynamical approach, such as evolutionary game theory, one is able to follow the self-organization process by which a population of individuals coordinates into a given behavior. Hoffman et al. [13] argue that evolutionary dynamics is a powerful tool for specifying changes in strategies over time in a population. Quijano et al. [28] addressed the advantages of evolutionary game theory in the role of population games and evolutionary dynamics in distributed control systems. Ficici et al. [29] present a game-theoretic investigation of selection methods used in evolutionary algorithms. The three main advantages of using EGT in engineering problems and an outstanding advantage of distributed population dynamics compared to distributed learning algorithms are described in Reference [28].

Evolutionary game has been successfully applied to the areas of Advanced Persistent Threats (APTs), evolving interactions between an attacker and a defender, detecting DDoS, and wireless sensor networks. Alabdel Abass et al. [10] studied APTs that represent stealthy, powerful, long-term, and well-funded attacks against cyber systems, such as smart grids, data centers, and cloud storage. The authors capture the long-term continuous behavior of the APTs on the cloud storage devices using evolutionary game. Bouhaddi et al. [30] model the evolving interactions between an attacker and a defender in MANET as an evolutionary game. In this model, each player learns about the behavior of its opponent over time and adjusts its strategy. Vejandla et al. [31] present evolving gaming strategies for attacker-defender in a simulated network environment. Detection of DDoS attacks using an artificial immune system-inspired multi-objective evolutionary algorithm has been investigated in Reference [32]. Evolutionary game theory has also been used for modeling wireless sensor networks [33,34,35,36,37,38]. An overview of evolutionary computation and other computational intelligence technology contributing to meet security challenges can be found in Reference [3].

Although our work is partly motivated by the related work above, there are some distinctions compared with them. The unique characteristics and usage scenarios of IoT-enabled AMI in the smart power grid introduce new security challenges. The increasing share of pervasive IoT devices which lack computing power, security, and privacy in such environments is a challenge—not to mention provisioning of adaptivity to tackle dynamicity and evolution. An accurate and resilient evolutionary game-based adaptive confidentiality assessment on IoT-enabled AMI entities is required. Given the dynamics in the AMI environment, the ability of the AMI nodes to adjust their confidentiality protection in response to their perception of the environment and the systems themselves should be provided. Although there are many research contributions about confidentiality in AMI systems, most of them have not considered these and fall short defining a framework for building dynamic and flexible defense for AMI with population dynamical methods for designing defense mechanisms for robust and reliable AMI cyber systems. Our work is similar to Ismail et al. [25]’s work in that it models the AMI as tree structure and in that it is a similar game model for confidentiality protection. It differs from their noncooperative classical game, which assumes player’s rationality, in that it is a population game with continuous game and no rationality assumption. The proposed solution is adaptive using the replicator dynamics for players to adapt their strategies, addresses the problem of evolutionary attacks, and uses a numerical simulation to solve the replicator dynamics.

Our analysis shows that the presented related work mostly studies scenarios where a single adversary attacks one resource at a time. In reality, multiple attackers can cooperate and launch joint attacks. They can share their knowledge about previous attacks, learn from each other’s experiences, and coordinate future actions by selecting successful strategies. The defenders can also collaborate and share the acquired experience to choose the optimal strategies for their defenses. We recognize that the collaborations between multiple adversaries and multiple defenders have not been fully researched in the previous work. This motivates us to study and to apply evolutionary game theory to these advanced scenarios to explore the space of strategies, as well as to select the optimal set of solutions. It will allow the defender to choose the best possible strategy and to continuously stay ahead of the attacker in defending the security intelligence.

3. Evolutionary Game Theory

In this section, we give an overview of evolutionary game theory that is further used in our formulation of the AMI confidentiality game. While classical game theory has been traditionally applied to model attacks on smart grid systems, it is a static approach that computes Nash equilibria and the corresponding utilities for the participating players. The main idea behind classical game theory is how rational individuals are expected to behave in conflict situations.

Formally defined, a game consists of N players and a strategy space S. Each player can select a strategy $s_i\in S_i\in S$, where $S_i$ is the strategy space of the i’th player. When a player selects a particular strategy, the corresponding payoff depends on this strategy and on decisions made by other players. The utility function is defined as ${\mathcal{U}}_i:S\to \mathbb{R}$, where $S=S_0\times S_1\times \cdots \times S_N$.

A Nash equilibrium (NE) is a strategy set $s^{*}\in S$ such that

$${\mathcal{U}}_i(s_i^{*},s_{-i}^{*})\ge {\mathcal{U}}_i(s_i,s_{-i}^{*})\forall i,s_i\in S,$$

(1)

where $s_{-i}\in S-S_i$, the strategy space excluding player i. In a Nash equilibrium, no single player can increase its utility by unilaterally changing strategy.

Certainly, the limitations, such as rational and static features, do not reflect the way the real world behaves in most situations. Inspired by the theory of evolution, evolutionary game theory [39] was introduced to overcome these limitations.

Evolutionary game theory borrows the notation from classical game theory, like strategy spaces, payoff matrices, and utility functions. Differently from classical game theory with its focus on rational individuals, evolutionary game theory considers populations of players that adopt various strategies and play contest against each other. It studies how successful these populations are in their choices of strategies and how more successful strategies are passed to the next generations. Therefore, it models the dynamics and evolution of populations of players given a distribution of strategies. Generations of population evolve based on the success of individual strategies compared to the success of overall population.

This evolution process is governed by two key elements:

• mutation mechanism that is represented by the Evolutionary Stable Strategy (ESS) concept; and
• selection mechanism that is represented by the replicator dynamics.

The ESS concept is considered to be a refinement of NE and it represents an ability to evolve. It outperforms any alternative mutant strategies. In other words, a strategy x is defined as an ESS if, for any other strategy y, some threshold fraction of mutants ${\overline{ϵ}}_y\in ]0,1[$ exists that Equation (2) is satisfied for all $ϵ\in ]0,{\overline{ϵ}}_y[$:

$$\mathcal{U}(x,ϵ\times y+(1-ϵ)\times x)\ge \mathcal{U}(y,ϵ\times y+(1-ϵ)\times x).$$

(2)

Thus, the strategy x is defined as evolutionary stable if this inequality holds for any mutant strategy, if the share of mutants in this population is sufficiently small [40]. A group of players choosing ESS will not be replaced by players that choose a different strategy. It is shown [40] that a strong connection between ESS and NE exists. If a strategy x is an ESS, then x is a Nash equilibrium, and, if x is a strict Nash equilibrium, then x is an ESS.

The second important concept is the replicator dynamics [41] that governs evolution of the strategies and is defined as following.

$$\frac{\partial x_i\left(t\right)}{\partial t}=(U\left(x_i\right)-U_A\left(x\right))\times x_i\left(t\right).$$

(3)

Here, $x_i$ is the proportion of strategy i in the population $x=(x_1,\dots ,x_n)$. $U\left(x_i\right)$ is defined as an expected utility of strategy i, and $U_A\left(x\right)$ is defined as an average population utility. Playing a game, different individuals from a population are able to compare their strategies to the average population result and learn from each others experiences. The replicator dynamics is applied to adjust their strategies. If ESS exists, the evolution dynamics leads to ESS [42].

Further improvement of the replicator equation was suggested in Reference [43]. The authors proposed to add stochastic elements to better address dynamic stability. In this work, we use the replicator equation with stochastic elements.

4. Models and Numerical Scheme Development

This section presents the system, threat, and game models for the AMI. Further, it defines the evolutionary game for confidentiality attacks and defenses based on these models. We propose a numerical scheme by discretizing the strategy spaces and deriving a discrete version of the replicator equation.

4.1. System and Threat Model

The work considers a network scenario where adversaries attack an AMI network trying to compromise confidentiality and obtain unauthorized access to the information transmitted inside the network. In this scenario, we assume that the adversary has knowledge about the network topology including transmission technology. Further, we assume that the system deploys an intrusion detection system (IDS) to detect malicious behavior.

The configuration of an AMI is modeled as a graph that connects nodes representing individual meters, collectors, and the head-end system. Further, we define a set of nodes $N=\{0,1,2,\dots ,n\}$ that comprise the AMI network. The head-end system (HES) node is defined as the top node $n_0$. We have two sets of nodes: collectors and meters. Meters collect the data and forward the data to the HES node using collectors as transmitters. Some nodes can perform as both meters and collectors. Due to the hierarchical nature of the information aggregation taking place, we represent the AMI as a tree structure, as shown in Figure 1. The tree is static, meaning that vertices and edges do not change over time. Except for the HES node that does not have a parent, the rest of the nodes have one parent and may have multiple children.

Confidential data is collected by meters, aggregated at meters/collectors, and transmitted by the network to reach the head-end system. The information sent from each node has a quantified value and is the sum of the value of the information gathered at the node and the value of the information collected from the node’s children.

We denote $f\left(i\right):\mathcal{N}\to \mathcal{N}$ as the parent of node i, and the set

$${\mathrm{Ch}}_i=\left\{j\in \mathcal{N}:f\left(j\right)=i\right\}$$

(4)

as the children of node i.

As all meters are leaf nodes, the set ${\mathrm{Ch}}_i$ is empty for these nodes.

The AMI nodes can run on different security levels determining the probability of protecting the data before transmitting. Due to its limited computational budget, an AMI node is not capable of protecting all messages. In addition, the AMI uses an IDS to detect possible attacks. Thus, these resources are also taking in consideration.

A set of adversary nodes exists that can connect to and attack the AMI. We assume that the attacker cannot access the cryptographic keys and has no control over the encryption process. To intercept a message generated by the meter $n_i$, the adversary node can attack either the meter or any of collectors that forward the message. Attacking a leaf node, or a meter, is less expensive than attacking a collector. However, a successful attack on a collector gives higher payoff. For each node, we define a probability for protecting the messages as $t_i$. For each adversary node j, we define a probabilty for attackings an AMI node $n_i$ as $s_{i,j}$. For each node, the costs of defense and attack are given as $c_i^d$ and $c_i^a$, respectively. The collected messages represent a certain value. For quantifying these values, we present an asset value $v_i$ for each node $n_i$. We assume that these values are constant over time.

4.2. Game Model

We consider the following confidentiality game that involves two classes of players, attackers, and defenders. The attackers and defenders meet pairwise and play the game. The attacker and defender do not have any knowledge of the opponent’s choices and choose their strategies simultaneously. The game is a discretization of a continuous game and can be considered as a special case of a resource constrained network security game [44].

The attackers choose attack rates (or probabilities) $s_i\ge 0$ for attacking the node labeled i. The attacks are assumed to be subject to the budget constraint

$$\sum _{i=1}^N{s}_i\le B_S,$$

(5)

where N is the total number of nodes in the tree, and $B_S$ is the attacker budget. Hence, the strategy space for the attacker is given by

$$S=\{s\in {[0,1]}^N:\sum _i{s}_i\le B_S\}.$$

(6)

Similarly, the defenders choose defense rates $t_i\ge 0$ for defending node i. The defense rate is the proportion of data that is encrypted before transmitting it to its parent node. In particular, for $t_i=1$, the data sent from node i is assumed impossible for the attacker to obtain. The defense rates are assumed subject to the budget constraint,

$$\sum _{i=1}^N{t}_i\le B_T,$$

(7)

where $B_T$ is the defender budget. Hence, the strategy space for the defender is given by

$$T=\{t\in {[0,1]}^N:\sum _i{t}_i\le B_T\}.$$

(8)

We model an AMI as a tree structure, and we assume that, in order to intercept data sent from node i, the attacker can choose to either attack node i directly or the parent node $f\left(i\right)$. We consider that the cost of attacking and encrypting data on node i are proportional to the value of the data.

The utility function ${\mathcal{U}}_A:S\times T\to \mathbb{R}$ for the attacking player is given by

$${\mathcal{U}}_A(s,t)=\sum _{i=1}^N(v_i(s_i+s_{f\left(i\right)})(1-t_i)-s_i{C}_{A,i})=\sum _{i=1}^N(v_i{s}_i(1-t_i)-s_i{C}_{A,i})+\sum _{i=1}^N\sum _{j\in {\mathrm{Ch}}_i}{v}_j{s}_i(1-t_j),$$

(9)

where $C_{A,i}$ is the cost of attacking node i, and $v_i$ is the value of the information collected at node i. Note that a factor appearing in the work of Ismail et al., $(1-a)$ where a is the detection rate, is omitted here for simplicity. It is a scaling of the node values and has no quantitative influence on the model.

Similarly, the utility function ${\mathcal{U}}_D:T\times S\to \mathbb{R}$ for the defending player is

$${\mathcal{U}}_D(t,s)=-\sum _{i=1}^N(v_i(s_i+s_{f\left(i\right)})(1-t_i)+t_i{C}_{D,i}=-\sum _{i=1}^N(v_i{s}_i(1-t_i)+t_i{C}_{D,i})-\sum _{i=1}^N\sum _{j\in {\mathrm{Ch}}_i}{v}_j{s}_i(1-t_j),$$

(10)

where $C_{D,i}$ is the cost of defending node i.

4.3. Evolutionary Game Formulation

In this section, we proceed with the main novel contribution of this work, the application of the model [25] in an evolutionary game. To this end, we assume that there are populations of attackers and defenders, represented by probability measures $P_s\left(S\right)$ and $P_t\left(T\right)$, respectively. The measures represent the distribution of the overall population over the attacker and defender strategy space. Different strategies of population evolution exist, and, for the current work, we use a replicator equation.

The replicator equation favors the choices of strategies which perform well (in terms of utility) relative to the overall population. Let

$${\pi }_A(s,P_t)={\int }_T{\mathcal{U}}_A(s,t^{\prime })P_t\left(\mathrm{d}{t}^{\prime }\right)$$

(11)

be the expected payoff for an attacker given a defender population $P_t$. We can formulate a similar payoff for the defending population, denoted D. For the defenders, we have the expected payoff

$${\pi }_D(t,P_s)={\int }_S{\mathcal{U}}_D(t,s^{\prime })P_s\left(\mathrm{d}{s}^{\prime }\right),$$

(12)

given an attacker population $P_s$. Note that Equations (11) and (12) form a couple set of equations defining the dynamics between the attacking and defending populations.

Equations (11) and (12) can be integrated over the attacking and defending populations, respectively, to yield the average payoff. Given the attacker and defender populations $P_s$ and $P_t$ we obtain

$${\pi }_A(P_s,P_t)={\int }_S{\pi }_A(s^{\prime },P_t)P_s\left(\mathrm{d}{s}^{\prime }\right)={\int }_S{\int }_T{\mathcal{U}}_A(s^{\prime },t^{\prime })P_t\left(\mathrm{d}{t}^{\prime }\right)P_s\left(\mathrm{d}{s}^{\prime }\right).$$

(13)

Similarly, the average payoff for a defender given the same populations is given by

$${\pi }_D(P_t,P_s)={\int }_T{\pi }_D(t^{\prime },P_s)P_t\left(\mathrm{d}{t}^{\prime }\right)={\int }_T{\int }_S{\mathcal{U}}_D(t^{\prime },s^{\prime })P_s\left(\mathrm{d}{s}^{\prime }\right)P_t\left(\mathrm{d}{t}^{\prime }\right).$$

(14)

We can combine Equations (11)–(14) with the evolutionary replicator dynamic with noise [43] to model the evolution of any set of strategies. For any subset of attacking strategies $\overline{S}\subset S$, the evolutionary replicator dynamic with noise [43] takes the form of

$$\frac{\mathrm{d}{P}_s}{\mathrm{d}\tau }\left(\overline{S}\right)={\int }_{\overline{S}}({\pi }_A(s^{\prime },P_t)-{\pi }_A(P_s,P_t))P_s\left(\mathrm{d}{s}^{\prime }\right)+{\delta }_S\left(\overline{S}\right),$$

(15)

for some time scale $\tau$ and a stochastic term ${\delta }_s\left(\overline{S}\right)$. Similarly, the evolution of a subset of defending strategies $\overline{T}\subset T$ is given by

$$\frac{\mathrm{d}{P}_t}{\mathrm{d}\tau }\left(\overline{T}\right)={\int }_{\overline{T}}({\pi }_D(t^{\prime },P_s)-{\pi }_D(P_t,P_s))P_t\left(\mathrm{d}{t}^{\prime }\right)+{\delta }_T\left(\overline{T}\right).$$

(16)

The Equations (15) and (16) fully govern the evolution of the attacker and defender populations. The noise terms ${\delta }_S\left(\overline{S}\right)$ and ${\delta }_T\left(\overline{T}\right)$ introduce random fluctuations in the evolution of the attacker and defender populations, respectively.

4.4. Numerical Scheme

We carry out numerical experiments in order to provide insight into the evolutionary dynamics of the confidentiality game and to demonstrate how it can be used in informing AMI security. To this end, we discretize the strategy spaces and derive a simple but computationally effective numerical scheme for the replicator equation.

To solve (15) and (16) numerically, we discretize the N-dimensional strategy spaces S and T. An attacking strategy is represented by

$$s^k=\left(\frac{k_1}{K},\cdots ,\frac{k_N}{K}\right),k_i\in \{0,\cdots ,K\}.$$

(17)

The i’th element of the vector $s^k$ represents the degree (or severity) of attack aimed at node i. We impose a budget constraint

$$\frac{1}{K}\sum _{i=1}^N{k}_i\le 1$$

(18)

to ensure that each attacker only has a finite amount of resources, i.e., cannot fully attack every node at once. Similarly, a defending strategy is represented discretely as

$$t^k=\left(\frac{k_1}{K},\cdots ,\frac{k_N}{K}\right),k_i\in \{0,\cdots ,K\}.$$

(19)

Similar to the attackers, the defenders also have a budget constraint

$$\frac{1}{K}\sum _{i=1}^N{k}_i\le Y$$

(20)

imposed on their valid strategy space. This constrain ensures that the defenders cannot fully defend every node in the network and need, therefore, to select the strategy that provide the highest possible payoff.

It is convenient to introduce the discretized strategy space, i.e., the list of all possible strategies that also adheres to the budget constraint. We can denote the discretized attacker strategy space as

$${\Omega }_S^K=\left\{k=(k_1,\cdots ,k_N):k_i\in \{1,\cdots ,K\},\frac{1}{K}\sum _{i=1}^N{k}_i\le B_S\right\},$$

(21)

and, similarly, the discretized defender strategy space as

$${\Omega }_T^K=\left\{k=(k_1,\cdots ,k_N):k_i\in \{1,\cdots ,K\},\frac{1}{K}\sum _{i=1}^N{k}_i\le B_T\right\}.$$

(22)

In the discrete setting the population of attackers and defenders can be represented as a probability distribution over the discrete strategy spaces. In particular, the population of attackers is represented by the probability distribution

$$P_s\left(s^k\right)\in [0,1],\sum _{k\in {\Omega }_S^K}{P}_s\left(s^k\right)=1,$$

(23)

and the defender population is represented as

$$P_t\left(t^k\right)\in [0,1],\sum _{k\in {\Omega }_T^K}{P}_t\left(t^k\right)=1.$$

(24)

Using this representation, the expected payoff for an attacker and defender can be written in the form

$${\pi }_A(s,P_t)=\sum _{k\in {\Omega }_T^K}{\mathcal{U}}_A\left(s,t^k\right)P_t\left(t^k\right)$$

(25)

and

$${\pi }_D(t,P_s)=\sum _{k\in {\Omega }_S^K}{\mathcal{U}}_D\left(t,s^k\right)P_s\left(s^k\right),$$

(26)

respectively. Similarly, the average payoff for the attacker and defender populations can then be written as

$${\sigma }_A(P_s,P_t)=\sum _{k\in {\Omega }_S^K}{\pi }_A(s^k,P_t)P_s\left(s^k\right)$$

(27)

and

$${\sigma }_D(P_t,P_s)=\sum _{k\in {\Omega }_T^K}{\pi }_D(t^k,P_{s}t)P_t\left(t^k\right),$$

(28)

respectively.

Finally, the discrete versions of the population replicator equations take the form

$$\frac{\mathrm{d}{P}_s}{\mathrm{d}\tau }\left(s^k\right)=({\pi }_A(s^k,P_t)-{\sigma }_A(P_s,P_t))P_s\left(s^k\right)+{\delta }_s^k$$

(29)

for the attackers and

$$\frac{\mathrm{d}{P}_t}{\mathrm{d}\tau }\left(t^k\right)=({\pi }_D(t^k,P_s)-{\sigma }_D(P_t,P_s))P_t\left(t^k\right)+{\delta }_t^k$$

(30)

for the defenders.

Given initial attacker and defender populations $P_s^{\left(0\right)}$ and $P_t^{\left(0\right)}$, respectively, the i’th generation of attackers and defenders, $P_s^{\left(i\right)}$ and $P_t^{\left(i\right)}$, are computed iteratively according to the scheme

$$P_s^{\left(i\right)}\left(s^k\right)=P_s^{(i-1)}\left(s^k\right)+\Delta \tau ({\pi }_A(s^k,P_t^{(i-1)})-{\sigma }_A(P_s^{(i-1)},P_t^{(i-1)}))P_s^{(i-1)}\left(s^k\right)+\Delta \tau {\delta }_S^k,$$

(31)

$$P_t^{\left(i\right)}\left(t^k\right)=P_t^{(i-1)}\left(t^k\right)+\Delta \tau ({\pi }_D(t^k,P_s^{(i-1)})-{\sigma }_D(P_t^{(i-1)},P_s^{(i-1)}))P_t^{(i-1)}\left(s^k\right)+\Delta \tau {\delta }_T^k,$$

(32)

where $\Delta \tau$ is the time step length between each generation. For numerical stability, positivity of $P_s^{\left(i\right)}$ and $P_t^{\left(i\right)}$ is enforced after each step, and the populations are re-normalized.

5. AMI Case Study

5.1. Simulation Setup

We consider a case study with a small but realistic AMI structure consisting of 15 nodes with edges shown in Figure 2 and node parameters given in

Table 1. Case study parameters.

Node	${\mathit{v}}_{\mathit{i}}$	${\mathit{C}}_{\mathit{A},\mathit{i}}$	${\mathit{C}}_{\mathit{D},\mathit{i}}$	${\mathit{s}}_1^{*}$	${\mathit{t}}_1^{*}$	${\mathit{s}}_2^{*}$	${\mathit{t}}_2^{*}$	${\mathit{s}}_3^{*}$	${\mathit{t}}_3^{*}$
#1	33.00	6.60	1.65	0.396246	0.51805	0.373106	0.524613	0.364412	0.536942
#2	15.00	3.00	0.75	0.072595	0.04662	0.09603	0.107789	0.089545	0.511544
#3	18.00	3.60	0.90	0.060055	0.05209	0.054538	0.48788	0.060831	0.515485
#4	3.00	0.60	0.15	0.042049	0.03001	0.043214	0.074903	0.045137	0.12042
#5	12.00	2.40	0.60	0.046897	0.029263	0.049434	0.07387	0.053685	0.119965
#6	9.00	1.80	0.45	0.044304	0.028329	0.046061	0.071233	0.049054	0.119792
#7	9.00	1.80	0.45	0.041939	0.018402	0.043156	0.065708	0.045231	0.119011
#8	3.00	0.60	0.15	0.026952	0.04108	0.026954	0.077835	0.026961	0.121616
#9	3.00	0.60	0.15	0.026952	0.04108	0.026954	0.077835	0.026961	0.121616
#10	3.00	0.60	0.15	0.026952	0.04108	0.026954	0.077835	0.026961	0.121616
#11	3.00	0.60	0.15	0.037328	0.035544	0.036896	0.074586	0.036151	0.119706
#12	3.00	0.60	0.15	0.037328	0.035544	0.036896	0.074586	0.036151	0.119706
#13	3.00	0.60	0.15	0.046802	0.027631	0.046602	0.070442	0.046307	0.117527
#14	3.00	0.60	0.15	0.046802	0.027631	0.046602	0.070442	0.046307	0.117527
#15	3.00	0.60	0.15	0.046802	0.027631	0.046602	0.070442	0.046307	0.117527

. All leaf nodes represent meters. The intermediate nodes between the head-end system and the leaf nodes act either as a pure collector aggregating data from its child nodes, or as a hybrid collector/meter. We assume that the value of the information aggregated at each node the sum of any information generated at the node (if it is a meter) and the information aggregated from child nodes. The cost weights of attacking and protecting data on a node i are set to 0.2 and 0.05, respectively. These weights are based upon the original work of Ismail et al. [25]. We used a simular network configuration. However, we removed some leaf nodes in order to limit the strategy space. Further, we investigated several options regarding the defender’s budget.

The attacker’s budget is set to $1.0$ and is the same value for all simulations. Regarding the defender’s budget, we vary the budget as $1.0$, $2.0$, and $3.0$. The simulation results are presented separately for different values of the defender’s budget.

5.2. Evaluation Metrics

To better interpret the results from the evolutionary game and the evolution of attacker and defender strategies over time, we consider the following generation-dependent game metrics. The population-averaged attack-rate for node i is given by

$$A_i=\sum _{k\in {\Omega }_s^K}{s}_i^k{P}_s\left(s^k\right).$$

(33)

Similarly, the population-average defense-rate for node i is given by:

$$D_i=\sum _{k\in {\Omega }_t^K}{t}_i^k{P}_t\left(t^k\right).$$

(34)

To assess the success of attackers and defenders in the current population, the time-evolution of the average payoff (27) and (28) are also monitored.

6. Results

The simulation results that show the evolution of average payoff for both defenders and attackers are depicted in Figure 3.

The attack and defense rates for the different nodes are depicted in Figure 4, Figure 5, Figure 6, Figure 7, Figure 8 and Figure 9, respectively, for the defender’s budget equal $1.0$, $2.0$, and $3.0$. At the start of the game, the attack and defense resources are equally destributed between the participating nodes. Initially, there is a transient phase where the attackers alternate between attacking the Head-End System and the intermediate collector nodes. The defender population responds by increasing the defense rate for the Head-End System to the point where the attackers, on average, give up this node and instead focus on the intermediate Collectors and Collector/Meters.

The results for the attack and defense rates show that both types of players favor nodes from a higher aggregation level, which increase their utilities. We also observe that when the defender’s budget increases the system distributes the new resources to the nodes that contribute more to the defender’s payoffs. On one hand, the leaf nodes are less valuable and, thus, less effort is exerted to protect these nodes. On the other hand, the attacker uses also less resource to attack the leafs since the attacker better benefit by attacking nodes from a higher aggregation level. For the same aggregation level, both players assign more resources to the nodes with higher aggregated value. The simulation graphs show that more effort was made to protect the node 3, which has higher resource value and also accommodates more children than the node 2. It is worth noting that the resources assigned to defend the nodes are not linearly proportional to their respective values and that this proportion also changes depending on the amount of the total budget assigned to the attacker.

Further, we can see that the system finds a short unstable equilibrium state that happens around generation 70 for 1 defender, generation 60 for 2 defenders, and generation 50 for 3 defenders. For different values of the defender’s budget, both graphs start to converge to a stable state after approximately 80–90 generations and remain stable for more than 100 generations. For the defender, it means that the system has defined the solution that gives the best response to the adaptive attacks in the dynamic environment. The system, therefore, provides the adaptivity by using the replicator dynamics. We can also conclude that the ESS is reached for this system setup.

7. Conclusions

In this work, we modeled confidentiality attacks and defenses as an evolutionary game and analyzed the behaviors of the attacker and the defender of the AMI system. By applying evolutionary game theory to this problem, we introduced an important dynamic and learning capabilities in the behavior of both attackers and AMI nodes, to explore the space of strategies, and to select the optimal set of solutions. We used the replicator equation to show the evolution of utilities for both type of players. Further, we outlined how the evolutionary game model can be used to evaluate the security threats in the AMI systems. In our simulation scenarios, we show that the solution converges to ESS for all investigated cases. The simulations also show that the behavior of the replicator dynamic depends not only on incentives but also on the network configuration and proportions of the protected assets. It is important that the results of this work give us the best possible defense strategy against evolving attacks. It allows the defender to to continuously stay ahead of the attacker in defending the AMI nodes.

The next step in our research will be the investigation of more sophisticated scenarios, including dynamic AMI trees. We consider that these trees can have a dynamic configuration and change over time. A node can disconnect, connect to a new one, or a new node can be introduced. This condition introduces more variety and dynamism to the AMI system. This problem should be taken into account for preparing the strategy spaces. Applying the developed solution to real networks for further field testing and validation will be part of the roadmap for implementing this solution as an industrial toolset. Combining the evolutionary game analysis with machine learning algorithms, especially with federated learning and autonomy, is a step forward to overcome this limitation and the scaling problem. In our future work, we intend to evaluate and implement a combination of applying machine learning and evolutionary game theory for modeling adaptive attack-defense dynamics. It will also include a development of suitable quantitative metrics to evaluate game simulations.