Study on Stochastic Differential Game Model in Network Attack and Defense

In recent years, evolutionary game theory has been gradually applied to analyze and predict network attack and defense for maintaining cybersecurity. -e traditional deterministic game model cannot accurately describe the process of actual network attack and defense due to changing in the set of attack-defense strategies and external factors (such as the operating environment of the system). In this paper, we construct a stochastic evolutionary game model by the stochastic differential equation with Markov property. -e evolutionary equilibrium solution of the model is found and the stability of the model is proved according to the knowledge of the stochastic differential equation. And we apply the explicit Euler numerical method to analyze the evolution of the strategy selection of the players for different problem situations. -e simulation results show that the stochastic evolutionary game model proposed in this paper can get a steady state and obtain the optimal defense strategy under the action of the stochastic disturbance factor. In addition, compared with other kinds of literature, we can conclude that the return on security investment of this model is better, and the strategy selection of the attackers and defenders in our model is more suitable for actual network attack and defense.


Introduction
With the development of the Internet, the security of the network and the privacy of users have been greatly disturbed. erefore, the issues of cybersecurity have caused people's high attention. e security of the Internet has become one of the important factors hindering the development of information technology. It is impossible to guarantee the security of cyberspace by relying on some passive defense measures in the increasing complexity of the network environment. erefore, it is especially necessary to find new technologies that can detect the potential danger of network environment and take defense measures.
In the network attack and defense, intruders can carry out an intrusion and the computer network can resist attack, which is similar to the process of the evolutionary game. erefore, quite a lot of research studies have established a network attack and defense game model to select the optimal strategy [1][2][3][4][5][6]. e study of game theory first appeared in the field of economics research. In 1944, John von Neumann and Oskar Morgenstern proposed "game theory and economics," which received wide attention [7]. Evolutionary game is a theory that combines game theory with the dynamic evolution process. It adopts the evolutionary theory of biology based on traditional game theory. e development of evolutionary game theory in various fields can be attributed to Smith (1973) and Price (1974) [8], who proposed the basic concept: Evolutionary Stable Strategy (ESS). Among them, the participants are the bounded rationality (between completely rational and incompletely rational). e players between groups constantly correct, imitate, and improve during the evolution process. ey gradually tend to a certain stability strategy and eventually reach a state of equilibrium in the game. And players get the best strategy (to maximize their profits) in this state. In the field of cybersecurity, the traditional evolutionary game model does not consider the external environment and strategy mutation, which leads to the limitation of the evolution trend. e prejudgment of network attack and defense is also not accurate enough. erefore, researchers tried to further improve the effectiveness of the model and more accurately describe the evolutionary game of attack and defense by using stochasticity [9][10][11][12][13][14][15][16][17][18][19][20][21][22][23][24][25]. In the stochastic game of this paper, the attackers will try to interfere or destroy the network environment. e defenders (network environment) can enhance the defensive ability by increasing defensive investments. Based on the principle of bounded rationality, the players gradually evolve into a stable state by learning and improving. e accuracy of the defenders' choice of the optimal strategy has been effectively improved and the security of cyberspace has been guaranteed. e main contributions of this paper are as follows.
(1) e network attack and defense stochastic game model is constructed under incompletely rational conditions. We use stochastic differential equations to consider the randomness caused by external factors in the process of attack-defense. And we construct the stochastic replication dynamic equation to further accurately describe the evolution of the network attack and defense strategy. (2) e attack lethality coefficient is used to describe the impact of different attack strategies on players. Furthermore, the model proposed in this paper is compared with other kinds of literature by the defenders' payoffs, which further proves that the model proposed in this paper is more suitable for the actual situation of the network attack and defense. (3) e selection algorithm of the optimal defense strategy under this model is designed. is algorithm can provide effective support for active defense in the process of network attack and defense. e remainder of this paper is organized as follows. Related work is discussed in Section 2. In Section 3, the network attack and defense stochastic differential game model and corresponding concepts are described and analyzed. In Section 4, the stochastic differential game optimal defense strategy algorithm is introduced. Simulation experiments and results analysis are presented in Section 5. Finally, this paper is concluded in Section 6.

Related Work
e application of evolutionary game theory in cybersecurity has become a study boom in recent years. In the actual attack and defense process, the change of the system operating environment and the disturbance of other external factors have stochasticity. erefore, researchers began to introduce stochastic evolutionary game theory into the study of cybersecurity. ere are two main aspects of concern: the first is to consider the offensive and defensive process as a random jump between multistates. e other is to construct a stochastic evolutionary game using stochastic differential equations.
In the analysis of the vulnerability of the network environment, the authors in [9] studied the security and reliability issues of software and hardware services. ey built the Markov chain to construct a stochastic evolution alliance game to evaluate the optimal strategy, and the model can be applied to various defensive scenarios in the cloud computing network. From the perspective of attack and defense, the authors in [10] utilized the game model to find network vulnerability state and established the mapping relationship between attack and defense states. ey quantified the level of network vulnerability and proposed a hidden Markov model. On this basis, they accurately inferred the attacking intent using the Viterbi algorithm. Govaert et al. modeled system dynamics as a discrete-time Markov process [11], which identifies equilibrium states and periods. And, in any initial state, it can converge to a balanced state for a limited time. e above literature [9][10][11] utilizes the Markov process, which can accurately characterize the stochastic behavior of the network system and the interrelationship between components. And it is convenient for calculating various safety targets. Wang et al. established an attack and defense game model based on stochastic Petri nets [12], which can analyze and evaluate the attack success rate, average attack time, vulnerable nodes, and potential attack paths of the target network. He et al. [13] defined the Stochastic Colored Petri Nets (SCPN) based on the Internet of ings (IoT) when studying the offensive and defensive scenarios of the smart home and obtained the game model of security situational awareness. It can effectively predict the attacker's potential attack strategy and achieve the purpose of promoting defense strategy selection. Talukder et al. [14] found that it is necessary to construct a suitable model to express the spread of threats in mobile IP. Talukder proposed four common mobile IP attacks and used SCPN as model, which effectively reduced the probability of successful attackers. To assess the risk of intrusion, El Bouchti and Nahhal [15] introduced the process and rules of constructing an SCPN model using attack trees and showed how to transform and analyze the attack tree in Stochastic Game Nets (SGN). Fanti et al. [16] proposed a network model of satellite base station (SBS) affected by attack and defense. e optimal defense strategy was obtained by calculating the Nash equilibrium, and the model was able to obtain the evolution equilibrium state under the stochastic game rules. e above literature [12][13][14][15][16] has a strong dynamic analysis ability for the concurrency, asynchrony, and uncertainty of the system. It has the advantages of less modeling language and intuitive graphical representation that can describe the state and behavior of the system. It has some functions that other methods do not have, such as system description, security analysis, and system testing. It can be accomplished graphically in the system model framework. However, these methods do not consider the issue of the participants' payoffs and costs.
Huang et al. [17] found that the attack and defense strategy usually changes dynamically and continuously. erefore, Huang used the Ito stochastic differential equation to construct a stochastic evolutionary game from the perspective of the actual attack and defense. e model accurately shows the evolution process of attack and defense by analyzing the continuous game evolution. e process of path discovery was modeled as a noncooperative stochastic evolution game when Wang et al. [18] studied radio network security. It was carried out by distributed strategy learning at each stage of the game process, which effectively bypassed the malicious nodes of the hybrid attack strategy. Wei et al. [19] designed the optimal load shedding technique to quantify the physical impact of the coordinated attack. For the interaction between attackers and defenders, the stochastic game model is proposed to select the optimal defense strategy and protect the network. e above literature [17][18][19] uses stochastic differential equations to describe the stochastic evolution process. It considers the direct impact of network security incidents and can effectively prevent malicious threats.
In addition, Riehl and Cao [20] introduced a hierarchical approximation algorithm while studying the stochastic evolutionary games. It can search the required strategies in stochastic evolutionary games and find the optimal results of the network attack and defense. Liu et al. [21] integrated multiple network security elements (such as assets, threats, and vulnerabilities) of multisensor mobile phones into standard data sets to improve the awareness of network security.
e Nash equilibrium of the hybrid strategy is calculated by the stochastic game model, and the security status of the network is evaluated effectively, comprehensively, and accurately. Subbulakshmi et al. [22] constructed a stochastic evolutionary game model to analyze the destructive techniques related to radio networks. e model evaluated the optimal solution to improve network performance. Kumar et al. [23] proposed a stochastic alliance game to realize data distribute in-vehicle network physical systems (VCPS) when studying the safety and comfort of the in-vehicle network. Vehicles can access various resources from the cloud environment. ese resources help people find optimized strategy selection by transmitting shortrange, medium-range, and remote information. Arfaoui et al. [24] proposed a stochastic game model to balance network performance and security. e model is more efficient than the basic algorithm in terms of network lifecycle and throughput. Chen and Yeh [25] discussed the robustness of noncooperative evolutionary game strategies from the perspective of stochastic Nash Equilibrium and then explored the application of stochastic evolutionary game theory.
In summary, the researchers regard the attack and defense process as the process of random hopping in multistate when using SCPN for modeling. It can better describe the offensive and defensive processes, but it is difficult to avoid the conditions that need to satisfy the complete information. Researchers constructed stochastic differential models with incomplete information by using stochastic differential equations, which can effectively describe the network attack and defense process. However, most literature is limited to a specific network environment for attack and defense confrontation, which leads to low versatility. Aiming at the existing research results, this paper studies and proposes a stochastic differential game model of network attack and defense that introduces the stochastic differential equation, and the model has Markov property. Based on the network attack and defense scenario, the evolution trend of the behavior strategies of the network attack and defense groups is analyzed. We find the optimal defense strategy and effectively analyze the behaviors of the attack and defense strategy on this basis.

Network Attack and Defense Stochastic
Differential Game Model e attack and defense groups choose different strategies for the game based on the incomplete rationality of attackers and defenders. Both sides constantly try to adjust and improve their decision-making methods in the process of attack and defense and form a new situation of the game finally. is process also highlights the dynamic equilibrium of evolutionary game theory.

Model Definition.
e players will always suffer from some uncertain factors in the actual network attack and defense. erefore, this paper defines the Network Attack-Defense Stochastic Differential Game Model (NADSDGM).

Definition 1.
e network attack and defense stochastic differential game model is defined as a quaternion model NADSDGM � (N, S, U, τ), where we have the following: (a) N � (N A , N D ) represents the players in the attack and defense evolution game, that is, the participants who adopt strategies in the game. e participants have different meanings in different environments. ey can represent individuals and can also represent a team or a group of multiple teams. Among them, N A are the attackers and N D are the defenders (defense system).
represents the payoffs of the attackers when the attack is not performed, and U 1 A represents the payoffs of the successful attack by the attackers.
. U 0 D represents the expected payoffs when the defenders do not make a defensive investment, and U 1 D represents the expected payoffs of the defenders after defensive investment. (d) τ � (τ 0 , τ 1 ) indicates stochasticity. Among them, τ 0 � 0 indicates that the model does not use stochastic disturbance factors. τ 1 � 1 indicates that the model uses stochastic disturbance factors.

Parameter Quantization.
In the analysis of the attack and defense evolution game, we first define some relevant parameters to be convenient for the quantification of the payoffs.

Security and Communication Networks
Definition 2. Attack cost C A : this indicates the financial and material resources that the attackers need to perform the attack. e defenders do not invest in a defensive strategy, whether the attackers can successfully implement the attack depends only on the defenders' system vulnerabilities, and the attack cost at this time is C 0 A . When the defenders make defensive investments, it will increase the attacking difficulty of the attackers, and the attack cost is C 1 Definition 3. Incentive mechanism remuneration R: this represents the third-party regulator's reward for the defenders. In today's information age, the degree of possession of information resources and monopoly determine benefits. e main reason why the target network is attacked is that the information is not public and opaque, and the attackers want to obtain certain information through the attack. erefore, social regulators use incentive mechanisms to motivate defenders to properly publish information and share resources without harming their interests. e society that benefits from this will also reward defenders. To reduce the damage caused by the defense system being attacked, the defenders choose appropriate public information to receive social rewards. e more beneficial the public information is to society, the more rewards the incentives will generate. R represents the remuneration for the incentive mechanism. e remuneration is R 0 when the defenders do not make a defensive investment. When the defenders make defensive investments, the remuneration is R 1 .
Definition 4. Penalty cost G: this means that the third-party regulator punishes attackers who have committed attacks.
Internet attacks can lead to a series of cybersecurity issues, such as the users' data being leaked and the network's services being forced to be interrupted. It affects people's daily work and life and even affects the country's safety in case of seriousness. erefore, it is the responsibility of the third-party regulatory authority to punish the attackers for violating the cybersecurity. G is used to indicate the punishment of the attackers by the supervisor. When the defenders do not make a defensive investment, the attackers receive the penalty of G 0 . Correspondingly, when the defenders take defensive investments, the attackers receive the penalty of G 1 .
Definition 5. Attack lethality coefficient λ and defender loss l.
In actual network attack and defense, for different attack strategies, the defenders' loss is affected by the lethality of the attack. Assume that the total loss caused by a network attack to the defenders is l. e more lethal the attack strategy is, the less likely the defenders are to resist successfully, and the greater the loss suffered, that is, the greater the loss suffered by the defenders. On the contrary, the weaker the lethality of the attack strategy, the smaller the loss suffered by the defenders.
Let attack lethality coefficient be Among them, m represents the attack dangerous level (m ∈ N * ), and the lethality coefficient of attack changes under the influence of the dangerous degree of attack strategy.
We define l � λL. When the attack dangerous level is not enough to hurt the defenders, the loss of the defenders is 0. When the attack dangerous level is at 1 ≤ k ≤ m, the defenders can take remedial measures in time to reduce part of the loss. At this time, the attack lethality coefficient is e − (1/k) , and the defenders' loss is l � e − (1/k) L. When the attack dangerous level is large enough, it can be regarded as attack lethality coefficient being 1 and defenders' loss being l � L. Definition 6. Total return E: this represents the total return that the attackers can obtain from a successful attack.
When the defenders adopt defensive investment strategies, the payoffs of the attackers' successful attack are When the defenders do not adopt defensive investment strategies, the payoffs of the attackers' successful attack are P 0 E − C 0 A − G 0 . Additionally, the defenders as the target network are also capable of a certain defense. But to protect their infrastructure and information assets from harm, defenders can choose to increase their defensive investments against network attacks. Assume that the defenders' original defensive infrastructure and informational assets are collectively called the original asset V 0 , and the investment cost per time is V add . erefore, before and after the defensive investment by the defenders, the losses caused by the network attack are P 0 l and P 1 l, respectively. When the attackers successfully attack, if the defenders choose defensive investment strategies, their expected payoffs are V 0 − V add − P 1 l + R 1 . And if the defenders do not adopt a defense investment strategy, the expected payoffs are V 0 − P 0 l + R 0 . On the contrary, when the attackers do not take any attack, the attackers' expected payoffs are 0. And the defenders' expected payoffs before and after the defensive e main parameters and descriptions involved above are shown in Table 1.

Stochastic Differential Equation.
Assume that, in the process of attack and defense games, the proportion of attack strategies adopted by the attackers' groups is x, and the proportion of adopting nonattack strategies is 1 − x. e proportion of defensive investment strategies and nondefensive investment strategies in the defenders' group is y and 1 − y, respectively. Using the above parameters, the payoff matrix of the network attack and defense evolutionary game model is shown in Table 2.
Use U 1 D to indicate the expected payoffs of the defenders when the defenders choose to invest in the defense strategy, and U 0 D indicates the expected payoffs of the defenders when the defenders do not invest in the defense strategy. From the above payoff matrix, we can know Use U D to indicate the average payoffs of the defenders, which can be obtained by equations (2) and (3) Correspondingly, U 1 A indicates the expected payoffs of the attackers when the attackers adopt the attack strategy, and U 0 A indicates the payoffs of the attackers when the attackers do not adopt the attack strategy; that is, Use U A to indicate the average payoffs of the attackers, which is available from equation (5) and (6) According to the above analysis, the replication dynamic equation of the offensive and defensive evolution game model is obtained. From (5) and (7), the attackers' replication dynamic equation is e defenders' replication dynamic equation is obtained by equation (2) and (4) In order to characterize stochastic disturbance factors, the common method is to add a stochastic disturbance after replication dynamic equation. It satisfies the Gaussian hypothesis and obeys the normal distribution, which can reflect the stochastic effects caused by many tiny factors. Common Markov processes include Poisson process and Wiener process, and white noise has become a kind of stochastic disturbance commonly used in system analysis [17]. erefore, this paper uses the white noise process as a stochastic disturbance in the game process, and (8) and (9) are modified to obtain where dθ 1 (t)dθ 2 (t) represents the Wiener process; it has Markov property.

Optimal Defense Strategy Selection
In this section, the evolutionary equilibrium solution and stability analysis of the stochastic equation are firstly proved, and then the optimal strategy selection algorithm is given.

Evolutionary Equilibrium Solution.
Because the stochastic game model proposed in this paper is composed of nonlinear Ito stochastic differential equations, the analytical solution of the equations cannot be obtained directly. erefore, in this section, we first prove that the stochastic differential equation presented in this paper has a unique solution (i.e., satisfying the local Lipschitz condition and the linear growth condition [26][27][28][29]). And in the following Proof. We rewrite equation (10) Among them, Obviously, f 1 (x, y), f 2 (x, y), g 1 (x, y), and g 2 (x, y) are continuous on [0, 1] × [0, 1].
In summary, equation (10) has a unique solution. □

Evolutionary Stability Analysis.
For the stochastic game model constructed, the stability of the game model is proved according to the conclusion described in [26][27][28][29], that is, the expected operation of the Ito integral and the exchangeable property of the integral operation.
Proof. Equations (11) and (12) are expressed as integral equations as where x(0) and y(0) are the values when t � 0.
Let m(x, y) � (1/|x(0)| 2 )e zt E(|x(t)| 2 ). Among them, E(Θ) is the expectation and z is the positively constant. en, Security and Communication Networks ereby, Among them, at is, for any X 0 , there is a constant of μ > 0, N > 0, and then the zero solution of equation (11) can be called the stability on the mean square exponential.
Similarly, for the zero solution of equation (12), the mean square exponential is also stable.

e Optimal Defense Strategy Selection Algorithm.
In the process of the network attack and defense, the attackers and the defenders play opposite to each other. Each player in the game is constantly testing, adjusting, and improving in the game to maximize their expected returns. Under the guidance of this principle, both attacker's strategy and defenders' strategy will gradually tend to balance. Neither party will try to change this strategy because the party that does not tend to balance will be reducing payoffs. at is to say, the strategy of achieving balance at this time is the optimal strategy. e specific Algorithm 1 is described as follows.

Simulation Results and Analysis
In this section, we first set up a network experimental environment. Due to the nonlinearity of the stochastic game model, the model is simulated by the explicit Euler numerical method.

Experimental Environment.
We deploy a network topology environment to simulate the network attack and defense evolution game model proposed in this paper. e validity of the model is proved by analyzing the evolutionary stability strategy.
As shown in Figure 1, in the network topology environment, attack host A is located on the external network and it is used to simulate a variety of attack strategies of attackers. e intranet contains three servers, namely, MySQL Server B, Web Server C, and FTP Server D. e internal network is isolated from the external network by the firewall.
Since the firewall separates the internal network from the external network, the external host can only access Web Server C and FTP Server D through the network. In the intranet, MySQL Server B, Web Server C, and FTP Server D can access each other by using user rights.
e Nessus vulnerability scanner is used to perform vulnerability scanning on three server nodes in the network. e server node information is shown in Table 3.
rough the analysis of the vulnerability and attack behavior of each host node in the network, combined with the China National Vulnerability Database of Information Security (CNNVD), the network attack and defense strategies are designed in the experiment, as shown in Tables 4 and 5. Assuming that the network attack strategy is S A1 and S A2 , the strategy S A1 has a high cost, high attack effectiveness, and strong pertinence. Strategy S A2 has low cost and low attack effectiveness, which can be considered as not attacking. In addition, assuming that the network defense strategy is S D1 and S D2 when defending against external attacks, the defenders can increase the cost to take defensive investments or can rely on the existing defense ability to passively defend. (10), we use the explicit Euler numerical method to simulate it [27]. N is the number of iterations, T is the game time, and the average step size is H � T/N.

Explicit Euler Numerical Results. For equation
Let N (0, 1) denote the standard normal distribution and divide t ∈ [0, T] into N ∈ N + equal parts; that is, interval [0, T] is divided into 0 � t 0 < t 1 < · · · < t N− 1 < t N � T, the average step size is H � T/N, and the node is t n � nH.

Attack-Defense Simulation and Analysis.
Stochastic evolutionary game is a kind of stochastic theory which combines game theory analysis with dynamic evolutionary process analysis. In the following, according to the problem situation of x and y, multiple simulation experiments are carried out on the constructed network environment. From the obtained simulation results, the dynamic evolution law of attackers x and defenders y can be analyzed intuitively; the prediction of attack and defense strategies can be realized. And the evolutionary stability strategy is found, that is, the optimal defense strategy in this state. In the simulation experiment, it is assumed that τ � 0 indicates the evolution of the attack and defense strategy without considering the stochastic disturbance factor. It is too ideal and the Input N A , N D who participated in the game and host node information. Output Attack strategy S * A , optimal defense strategy S * D . Begin (1) Initialize NADSDGM � (N, S, U, τ) / * Initialize stochastic evolutionary game model * / (2) Construct x, y/ * Construct the group probability of the selected strategy set of both attack and defense * / (3) Constructing a stochastic evolution game matrix between attack and defense (4) Construct the stochastic differential equation of the attackers and defenders, and see equation (10)    stochastic disturbance in the actual attack and defense is not solved. τ � 1 indicates that the game evolution after considering the stochastic disturbance factor is more realistic and more effective. e problem situation is x � 0.4, y � 0.7; that is, the attackers in the group select the hybrid strategy S A1 , S A2 with the probability of {0.4, 0.6}, and the defenders in the group select the hybrid strategy S D1 , S D2 with the probability of {0.7, 0.3}. It can be seen from Figure 2 that, after continuous evolution, the probability of the attackers selecting the strategy S A1 gradually tends to 0 and the probability that the defenders select the strategy S D1 gradually tends to 1. Both of them reach an evolutionarily stable state. e optimal defense strategy at this time is S D1 . erefore, in this situation, the defenders belong to a more active state of defense. e defense groups are willing to adopt defensive investment strategies for its vulnerability, and it is gradually increasing. e attacker groups gradually turn to the passive state of not taking the attack. e network environment is safer. Figure 3 shows the experimental results obtained when the problem situation is x � 0.5 and y � 0.6. e situation indicates that the attackers in the group select the hybrid strategy S A1 , S A2 with the probability of {0.5, 0.5} and the defenders select the hybrid strategy S D1 , S D2 with the probability of {0.6, 0.4}. As shown in Figure 3, after continuous evolution, the probability that the attackers finally select the attack strategy S A1 gradually tends to 1 and the probability that the defenders select the defense strategy S D1 gradually tends to 1. Both of them reach an evolutionarily stable state, and the optimal defense strategy at this time is S D1 . Analysis of the situation at this moment shows that the attackers and the defenders are actively adopting strategies to participate in the game; the network environment is in a relatively fierce state. e problem situation is x � 0.4 and y � 0.3; that is, the attackers in the group select the hybrid strategy S A1 , S A2 with the probability of {0.4, 0.6}, and the defenders select the hybrid strategy S D1 , S D2 with the probability of {0.3, 0.7}. After continuous evolution, the probability that the attackers finally select the attack strategy S A1 gradually tends to 0 and the probability that the defenders select the defense strategy S D1 gradually approaches 0. Both of them reach an evolutionarily stable state. e optimal defense strategy at this moment is S D2 . Figure 4 is a figure of experimental results in the situation of this problem. Analysis of the situation currently shows that although the network environment is relatively stable, the state of both offense and defense is relatively negative. e problem situation is x � 0.7 and y � 0.2; that is, the attackers select the hybrid strategy S A1 , S A2 with the probability of {0.7, 0.3}, and the defenders select the hybrid strategy S D1 , S D2 with the probability of {0.2, 0.8}. e experimental results obtained in this situation are shown in Figure 5. It can be observed from Figure 5 that, after continuous evolution, the probability that the attackers finally select the attack strategy S A1 gradually tends to 1 and the probability that the defenders select the defense strategy S D1 gradually approaches 0. Both of them reach an evolutionarily stable state. At this moment, the optimal defense strategy is S D2 . In summary, the analysis of the situation at this time shows that the defenders choose defensive investment strategies with a small probability. It is more passive in the offensive and defensive confrontation, and the attackers gradually adopt effective attack strategies; the overall network environment is paralyzed. Figure 6 shows the effect of the attack dangerous level on attack strategies. As we can see from Figure 6, when the attack strategy S A1 is not dangerous enough to hurt the defenders (that is, when k � 0 or 1), after the evolution equilibrium is reached, the probability that the attackers continue to select the strategy S A1 is about 0. at is to say, the attackers tend not to adopt the strategy S A1 . When the attack strategy S A1 is more dangerous (that is, when k � 55), the defenders suffered losses but were not fatal. After the evolution equilibrium is reached, the probability that the attackers continue to select the strategy S A1 is about 0.3. When the attack dangerous level is k � m, the strategy S A1 is lethal to the defenders. At this time, the attackers' payoffs increase; the probability that the attackers continue to select the strategy S A1 is about 0.9. Figure 7 is the effect of the attack dangerous level on the defense strategy. As shown in Figure 7, regardless of how many times the attackers use the strategy S A1 , the defenders actively select the strategy S D1 to respond. However, when k � 55, after about 0.2 h, the probability that the defenders choose the strategy S D1 to deal with is gradually less than 1. is is because the strategy S A1 does less damage to the defenders, and the attackers gradually choose not to adopt the strategy S A1 . Accordingly, the defenders also began to show that they did not adopt the strategy S D1 .

Comparison Consequence with Other Literatures.
Compared with other kinds of literature, we introduce the concept of the Return on Security Investment (ROSI) to measure the effectiveness of the attack and defense game model. ROSI is an important benchmark to decide the  optimal security investment level; researchers have used ROSI to measure the benefits of defenders. According to the Sonnenreich equation [30], we can get ROSI of attack and defense game model. Figure 8 is a comparison of ROSI. As shown in Figure 8, we can draw a conclusion that ROSI of literature [4] and this paper are better and more suitable for the real network attack and defense environment.
In addition, we also made a comprehensive comparison with some typical research results; as shown in Table 6, we can see that the traditional game model constructed in [1]   dynamic but not as good as the evolutionary game. e literature [4] adopts the evolutionary game. It has good versatility, but it is difficult to accurately describe the evolution process of attack and defense because the model does not consider stochasticity. e literature [6] adopts dynamic detection game, which improves the APT (Advanced Persistent reats) detection performance in the dynamic games and has better data protection ability, but it does not consider the influence of stochasticity on strategy and its application field is data protection. e literature [12] regards the offensive and defensive evolution game as the random jump process of multistate, but the condition of complete information is challenging to meet in the actual network attack and defense. e literature [9] considers stochasticity, but the model has a small scope of application and its versatility in general. In this paper, the stochasticity of the model is considered based on the condition of incomplete information, and the model is constructed by using stochastic differential equations, which improves the effectiveness of the model.

Conclusion
Nowadays, the analysis method based on the traditional dynamic game cannot meet the actual demand. In this paper, we construct a stochastic differential game model in network attack and defense by using stochastic differential equations based on Markov property. In different problem situations, the attackers and defenders will eventually tend to a stable state via continuous evolution. Compared with the strategy model without considering stochastic factors, it is proved that the model proposed in this paper is more suitable for the actual network attack and defense. By comparison, we can intuitively find that the theoretical analysis is consistent with the conclusions obtained by the simulation experiment, which proves the significance of the attack and defense evolutionary game model proposed in this paper. Compared with other related kinds of literature, we can conclude that the return on security investment of this model is better. Applying the model to the actual network environment can provide the choice of the defenders' optimal defense strategy and have a certain positive effect on the maintenance of cybersecurity.

Data Availability
e data used to support the findings of this study are available from the corresponding author upon request.  Defender's profits Literature [1] Literature [4] Literature [5] This paper  Dynamic game Incomplete information General General Strategy selection [6] Dynamic game Incomplete information Good Good Data protection [4] Evolutionary game Incomplete information Good General Security defense [12] Stochastic game Complete information Good General Security defense [9] Stochastic evolutionary game Incomplete information General Good Strategy selection is paper Stochastic differential game Incomplete information Good Good Strategy selection 14 Security and Communication Networks