^{1}

^{2}

^{1}

^{3}

^{1}

^{4}

^{1}

^{2}

^{3}

^{4}

In this paper, the game theoretical analysis method is presented to provide optimal strategies for anomaly-based intrusion detection systems (A-IDS). A two-stage game model is established to represent the interactions between the attackers and defenders. In the first stage, the players decide to do actions or keep silence, and in the second stage, attack intensity and detection threshold are considered as two important strategic variables for the attackers and defenders, respectively. The existence, uniqueness, and explicit computation of the Nash equilibrium are analyzed and obtained by considering six different scenarios, from which the optimal detection and attack actions are provided. Numerical examples are provided to validate our theoretical results.

Nowadays, network devices and communication services are vulnerable to various kinds of intrusion attacks, such as DoS/DDoS, false data injection, and botnet attacks. The intrusion attacks tend to be more intelligent and the unexpected attack modes arise frequently. Consequently, great challenges are brought into network security control and management. As one of the most important techniques to tackle with various attacks, anomaly-based intrusion detection system (A-IDS) has been widely adopted in almost all kinds of network environments [

In an intrusion detection system, the attacker and defender can naturally be regarded as two players who try to maximize their payoffs, respectively, by executing certain optimal strategies. Thus, the game theoretical method is an effective tool which enables a defender to earn the maximum payoff (or the minimum loss) while fighting with the attacks. A number of results on game theory-based intrusion detection methods have been reported for different network environments and security requirements. Excellent surveys about this topic can be found in [

In a game between an attacker and a defender, the rational attacker will not launch an attack otherwise she/he can get a positive payoff. Moreover, the attack intensity needs be chosen to maximize her/his positive payoff. On the contrary, the defender will perform a defense action to resist the attack according to a similar rule. In an A-IDS, a predefined detection threshold needs be cautiously determined. In general, a higher threshold with a larger normal coverage area will result in a smaller false alarm rate but a larger missing alarm rate. Note that the missing alarm rate is also closely related to the attack intensity. More specifically, larger attack intensity will cause a lower missing alarm rate. Though attack intensity and detection threshold are two important factors affecting the false and missing alarm rates, which correspond to the payoffs of attackers and defenders in an intrusion detection game, they are seldom considered in the aforementioned results. In most of the aforementioned works, the false and missing alarm rates are assumed to be known constants and only binary actions “do” or “not do” are considered in their game models. In [

Motivated by the limitations mentioned above in the literature, a more realistic two-stage form of the intrusion detection game model is presented in this paper. The attack intensity and detection threshold are considered as two strategic variables. In the first stage, the attackers and defenders make decisions on whether the attack and defense actions should be executed, respectively. Once the attack/monitoring actions are decided to be executed, optimal attack intensity and detection threshold are determined to maximize their utilities in the second stage. The existence and uniqueness of the Nash equilibrium are discussed for the first stage of our presented game model under different scenarios, when the strategic variables of the second stage are restricted to certain regions. Then, the optimal attack intensity and detection threshold are derived for each scenario, correspondingly.

The contributions of this paper can be summarized as follows:

A two-stage game model is presented for anomaly-based intrusion detection confrontation. In contrast to the existing work, where only binary actions “do” or “not do” are considered in the game model, the attack intensity and the detection threshold are considered as two key strategic variables, and the false and missing alarm rates are the functions of the attack intensity and the detection threshold, instead of being assumed to be constant. The two stages of the game model are tightly coupled with each other and thus the game model is more complex.

The existence, uniqueness, and calculation of Nash equilibriums are discussed. Based on the results, optimal selections of attack intensity and detection threshold for achieving the maximum payoffs of the attackers and defenders are provided. The results provide a new method to determine the detection threshold in the defense, from the perspectives of the optimization and confrontation. So, the presented game model and Nash equilibrium solution give a more realistic theoretical analysis framework for the anomaly-based security detection.

The rest of this paper is organized as follows. In Section

Suppose that there is a network unit vulnerable to intrusion attacks. Typical examples for such a unit include a software system, network equipment, and a communication channel. Here, we adopt similar attack and A-IDS detection models as that in [

Strategic form of the local game.

Monitor | Not monitor | |
---|---|---|

Attack | ||

Not attack |

In the following, we give the physical meanings of the corresponding variables in Table

Clearly, the game model described in Table

Two-stage pure-strategic intrusion detection game.

Players | Attacker, defender |
---|---|

Strategy sets | Attacker: Attack, not attack, attack intensity |

Defender: Monitor, not monitor, detection threshold | |

Constraints | |

Payoffs | |

Game target | The players choose their strategies to maximize their payoffs |

The attack and detection models are similar to that in [

As mentioned in Section

Denote the feasible set of

It can be readily shown that

For convenient expression in what follows, two variables

The optimization problems presented by (

Only one of the subsets

The following conclusions can be drawn.

In Scenario L.1, the NE of the game, as described in Table

If only the subset

If only the subset

If only the subset

If only the subset

Firstly, the strategy combination “attack, not monitor” will not be the NE. This is because,

If only

If only

If only

Based on the property that

If only

From Theorem

As discussed previously, Scenarios L.2∼L.5 cover the possibilities that

The following results about the NE for this scenario can be shown.

In Scenario L.2, the strategy combination “attack, not monitor” is the unique NE and

The subset

Main results for this scenario are formally stated in the following theorem.

In Scenario L.3, the strategy combination “attack, monitor” is the unique NE if and only if

Necessity: if “attack, monitor” is the unique NE, then from (

Sufficiency: since

The following conclusions can be drawn for this scenario.

In Scenario L.4,

If and only if

If and only if

Necessity: under the strategy combination “attack, not monitor”, the attacker will select

Sufficiency: from the definitions of

This means when the attacker selects

Necessity: under the strategy combination “attack, monitor,” the defender and attacker will select

This means the defender never earns larger payoffs than she/he does nothing, which is a contradiction to the premise that “attack, monitor” is the NE. Thus, the necessity is shown.

Sufficiency: the attacker always selects “attack” from the definitions of

Based on Theorem

In Scenario L.4,

If and only if

If and only if

From Theorem

Different from Scenario L.4, there exists

In Scenario L.5,

If and only if

If and only if

If and only if

If and only if

The proof is similar to that of (1) in Theorem

Different from Scenario L.4, there exists

(3) and (4) By following similar analysis in the proof of Corollary

In contrast to previous scenarios,

From (4) in Theorem

In Scenario L.6, the NE for the game as described in Table

If

If {

If {

If {

As there exists an

If

When

Firstly, if

It can be seen from (3) in Theorem

In this paper, we assume that the attackers are completely rational, while this assumption may not be satisfied in some scenarios. However, based on our method, we present an optimal defense strategy for the worst case. That is, we can guarantee that the maximum damage in the worst case can be minimized by our method.

In this section, simulation results are provided to validate the theoretical results as presented above. In A-IDS, a profile is generally selected to cause distinctions between normal and abnormal states. Such a profile is normally described by a random variable in many cases. Here, we assume it follows a Gaussian distribution with zero mean under normal states. Similar assumptions can be seen in many intrusion detection application areas such as network traffic detection and Kalman filtering-based anomaly detection. Let the intensity of the attack be denoted as

We first select

If

If

If

Then, it can be checked whether the inequality conditions in Theorems

Based on (2) in Theorem

Based on (2) in Theorem

Based on (4) in Theorem

The results showing whether the inequality conditions in Theorems

Scenario L.4 | Scenario L.5 | Scenario L.6 | Scenario L.6 | ||||

IC 4.1 | × | IC 5.1 | × | IC 5.1 | × | IC 5.1 | × |

IC 4.2 | √ | IC 5.2 | √ | IC 5.2 | √ | IC 5.2 | × |

Payoff of the attacker

In this case, we fix

If

If

Similarly, Table

Based on Theorem

Based on (2) in Theorem

Therefore, “attack, monitor” is always the unique NE if

At last, we make some comparisons with the existing methods in [

It can be seen that, without considering the attack intensity and detection threshold, the payoffs of the game model will be reduced to be constant and the Nash equilibrium analysis can be easily done. From the definition of the Nash equilibrium, it can be calculated that if

The results about the inequality conditions in Theorems

Scenario L.4 | Scenario L.5 | ||

IC 3 | √ | IC 5.1 | × |

IC 5.2 | √ |

Payoff of the defender

Strategic form of the game in existing work.

Monitor | Not monitor | |
---|---|---|

Attack | ||

Not attack | ||

For anomaly-based intrusion detection system, we present a game theoretical analysis method to provide the optimal strategies. We first establish a more realistic game model by considering the attack intensity and detection threshold as two strategies for the players. The necessary and sufficient conditions, for which strategies are the Nash equilibriums, are presented. Simulation studies are provided to validate our theoretical results. The results provide a new method to determine the detection threshold in the security defense. In the future, some more research work could be considered, for example, the game theoretical analysis method for specific scenarios such as Internet of Things and DoS/DDoS attacks. Besides, dynamic game analysis is also an interesting topic for dynamic security confrontation process, for example, Stackelberg game analysis can be adopted to solve the sequential problem of the attack and defense actions.

The manuscripts of game theory algorithm in this article are from the databases of Cambridge University and Columbia University. Copies of these data can be obtained from

The authors declared that they have no conflicts of interest.

This work was supported by the Basic Scientific Research Projects of National Defense Science, Technology and Industry Technology under Grant no. JSZL2017601C-1 and in part by the National Natural Science Foundation of China under Grant nos. 61897069 and 61831003, National Key Research and Development Program of China under Grant no. 2017YFB0801903, and National Key Program for Basic Research of China under Grant no. 2017-JSJQ-ZD-043.