One of the key advantages of unmanned swarm operation is its autonomous cooperation. When the communication is interrupted or the centralized control manner is lost, the cooperative operation can still be carried out orderly. This work proposed a cooperative evolution mechanism within the framework of multiplayer public goods game to solve the problem of autonomous collaboration of unmanned swarm in case of failure of centralized control. It starts with the requirement analysis of autonomous cooperation in unmanned swarm, and then, the evolutionary game model of multiplayer public goods based on aspiration-driven dynamics is established. On this basis, the average abundance function is constructed by theoretical derivation, and furthermore, the influence of cost, multiplication factor, and aspiration level on the average abundance is simulated. Finally, the evolutionary mechanism of parameter adjustment in swarm cooperation is revealed via case study, and deliberate proposals are suggested to provide a meaningful exploration in the actual control of unmanned swarm cooperation.
National Natural Science Foundation of China71901217National Key R&D Program of China2018YFC0806900China Postdoctoral Science Foundation2018M633757Primary Research and Development Plan of Jiangsu ProvinceBE20197621. Introduction
With the continuous advancing of the third wave of artificial intelligence, “group evolutionary intelligence” developed from “single-agent autonomous intelligence” has become one of the important characteristics of the new generation of artificial intelligence. Particularly in the military field, unmanned swarm (unmanned vehicle cluster [1], unmanned boat cluster [2], and unmanned aerial vehicle cluster [3]) operations have received unprecedented attention over the past two years. The US military has listed unmanned swarm operations as a “subversive technology” that can change the rules of war.
There are mainly two kinds of control modes of unmanned swarm: centralized control and autonomous collaboration. Under the premise of good communication, the Command and Control (C2) center can implement centralized control on the swarm. However, in the complex electromagnetic environment of the battlefield, there is a real risk of communication failure [4]. In such a predicament, the centralized control mode fails, and the unmanned swarm must make effective response on the spot according to the external situation and achieve self-management and self-coordination. An issue that has led to considerable interest is how unmanned swarms autonomously and cooperatively complete established military operations. The sketch of autonomous cooperation of unmanned swarm is shown in Figure 1.
Sketch map of autonomous cooperation of unmanned swarm.
Overall planning and reallocation of operation resources (communication, firepower, intelligence, etc.) within the unmanned swarms is required when autonomous collaboration occurs. However, in the process mentioned above, there are often contradictions between individual partiality and swarm needs, which are difficult to reconcile. For example, in the fire strike task, the “rational” unmanned units with intelligence and decision-making ability will choose to “contribute” ammunition to the swarm as little as possible in order to maintain its combat effectiveness, while on the other hand, the more ammunition each unit contributes to the swarm, the higher the survival rate and the greater the combat effectiveness of the whole swarm will be. The contradiction between the two will lead to “tragedy of the commons” [5]; therefore, how to increase the number of units’ willing to positively contribute ammunition to the swarm and avoid the tragedy has become a crucial and urgent problem in both technology research and practical application of unmanned swarm.
2. Related Work
Evolutionary game theory [6–9] combines “equilibrium” in economics with “adaptability” in biology to depict the process that individuals adapt to the external environment through learning, imitation, and trial-and-error under boundary rationality and asymmetric information. And the evolutionary game of public goods (PGG) [10] provides a basic theoretical framework for revealing the cooperative evolution mechanism and coping with the tragedy of the commons. PGG reflects that investors (collaborators) and hitchhikers (betrayers) play strategic games over time based on cost, multiplication factor, selection intensity, etc., which makes the proportion of collaborators and betrayers in the population change dynamically, and finally tends to an evolutionarily stable state (ESS). The research focus of PGG is to calculate the mathematical expectation of the proportion of collaborators in a population after multiround game, that is, the average abundance, and then analyze the relationship between average abundance and parameters (cost, multiplication factor, selection intensity, etc.) to achieve the ultimate purpose of manual control.
At present, there are two main research directions to solve the problem of swarm cooperation with evolutionary game theory: one is to study the evolutionary dynamics process and cooperation mechanism of spatial structured population such as complex network based on graph theory [11, 12], and the other is to study the evolutionary stability state of well-mixed population and the dominant condition of cooperation based on Markov stochastic process [13, 14].
For the former, the team of Professor Nowak from Harvard University theoretically deduced the evolution of population in spatial structure such as circle, random graph, and scale-free network and creatively proposed the relationship between the ratio b/c and the network average degree k. They pointed out that the smaller the network connectivity is, the more conducive the cooperation in natural selection is [15]. Then, they used the pair approximation theory to theoretically deduce the cooperation phenomenon on the regular lattice and obtained the boundary conditions for the generation and expansion of cooperation [16]. On the basis of the above achievements, further comparative analysis was made on the differences between homogeneous and heterogeneous networks in promoting cooperative behavior, and simulation results show that weak connection can better promote cooperative behavior on heterogeneous networks [17]. At the same time, other researchers studied the dynamic process of multiparty game on the graph, and simulation results show that spatial structured population can promote the occurrence of cooperation better compared with unstructured population. In the recent two years, the team of Nowak has applied the evolution dynamics of cooperation in spatial structure to social network, analyzed the critical conditions of cooperation behavior in human society [18], initially explored the trade-off between the evolution convergence probability and the evolution convergence time [19], and extended the cooperative evolution on structural population to weighted graph [20]. Other representative studies include literature [21, 22] on the specific model of multiplayer snowdrift game, the relationship curves between the ratio b/c and cooperation level on the well-mixed population and the structured population, respectively, and the significant differences between the homogeneous/heterogeneous network and the unstructured population in the promotion of cooperation are compared.
For the latter, the representative researches are as follows: Wang at Wuhan University obtained the average abundance function of the snowdrift evolutionary game using the stable distribution of Markov chain and simulated the effect of parameters on the average abundance [23]. Based on the research work of Tarnita et al. [24], Du at Peking University obtained the inequality of strategic dominance conditions in two-party evolutionary game through strict mathematical derivation; besides, the simulation results show that the average abundance under weak selection intensity is independent of aspiration level [25].
The aforementioned work on swarm cooperation is of great theoretical and engineering value. We have also conducted an exploration on the cooperation mechanism of unmanned swarm, and the relevant results can be referred to in [26–30]. Nevertheless, there are still two shortcomings in the above achievements in solving the issue of cooperative evolution of unmanned swarms: first, it has not focused on the public goods game; that is, although there are similarities between the snowdrift game and public goods game [31], there are essential differences in the game mechanism; in addition, the cooperative evolution of unmanned swarm is the multiple interaction of combat units; that is, the evolution result is not only related to the strategy selection of single unit, but also depends on the strategy of other units in the swarm, which is characterized by multiplayer games [32, 33]. So far, the academic community has mastered the payoff matrix [34] of the public goods game with multiplayer and has simulated the influence of different selection intensity [35–37] and threshold values [31] on the cooperation level. In particular, in literature [34], the authors derived a general average abundance formula of multiparty games in a finite population under aspiration-driven dynamics, which can be applicable to any multiparty game under the aspiration-driven dynamics of a finite population. However, the average abundance formula for specific public goods game is not mentioned, so the important work of this study is to obtain the average abundance analytic expression of public goods game based on the existing payment matrix.
Furthermore, the generalized evolutionary game model can be simplified as Markov chain + strategy update rules in finite population, so the average abundance function is also closely related to the strategy update rules. By studying strategy update rules in the framework of evolutionary game theory, one can differentiate between imitation processes and aspiration-driven dynamics [38]. In the former case, individuals imitate the strategy of a more successful peer [39]. In the latter case, individuals adjust their strategies based on a comparison between their own payoff and the value they aspire, called the level of aspiration [40]. Unlike the imitation processes of pairwise comparison, aspiration-driven updates do not require additional information about the strategic environment and can thus be interpreted as being more spontaneous [41, 42]. In the complex battlefield environment, the information acquisition is incomplete, asymmetric, which requires the swarm to achieve self-management and self-coordination, and this requirement just coincides with the aspiration-driven dynamics. Moreover, the existing results show that, in both prisoner’s dilemma game and public goods game, the dynamic mechanism driven by aspiration can improve the average abundance value and promote cooperation more than the traditional imitation dynamics [43, 44].
Aiming at the cooperative evolution mechanism of the unmanned swarm, we modelled the evolution process based on multiplayer public goods game framework and aspiration-driven update rule and then deduced the average abundance function of the model by analyzing the stable distribution of the Markov chain; on this foundation, we studied the influence of relevant parameters on the average abundance through theoretical analysis and numerical calculation; finally, we studied the effect of parameter adjusting on swarm cooperation via case study and discussed the corresponding solutions and advice to avoid “the tragedy of the commons.”
3. Model Hypothesis
In essence, the autonomous collaboration of unmanned swarms is a game process of multiparty and multiround, which focuses on the autonomous allocation of public resources. Therefore, we use multiplayer public goods evolutionary game to model the cooperative evolution of unmanned swarms. The mapping between the concepts of cooperative evolution in unmanned swarms and multiplayer public goods evolutionary game is listed in Table 1.
Mapping between the concepts of cooperative evolution and multiplayer public goods game.
Cooperative evolution in unmanned swarms
Multiplayer public goods evolutionary game
Unmanned swarms
Mixed homogeneous population
Public operational resources (communication, firepower, etc.)
Public goods
Multiple unmanned units participating in autonomous collaboration
Multiplayer
Single combat unit
Individuals
Single combat unit as research object
Focal individual
Investors (collaborators)
Strategy A
Hitchhikers (betrayers)
Strategy B
Total resources that individuals ultimately obtain after each round of game
Payoffs
Payoff-based strategy transformation between units
Game
Dynamic change of proportion of units with different strategies in multiround game
Evolutionary
The proportion is stable and the game ends after multiround game
Evolutionary stable state
3.1. Framework of Multiplayer Public Goods Evolutionary Game
It is set that the autonomous cooperation of unmanned units takes place in a well-mixed swarm of size N, and every unit has two alternative strategies, A and B. Every d units interact simultaneously to get their payoffs; i.e., they are in a two-strategy and d-player game. The strategy update procedure is as follows:
Select any focal individual X in the population of N and select d−1 individuals from the remaining N−1 individuals to form a group of dd<N. A focal individual can be of type A or B, and encounters a group containing k0≤k≤d−1 other players of type A and d−k−1 players of type B.
The focal individual X plays games with the rest d−1 individuals in strategy space A,B. Denote by ak and bk the payoffs of strategy A and B a player obtains, respectively, when facing i other A individuals within the group of size d, where i ranges from 0 to d−1.
At the end of each round of the game, the focal individual X evaluates the benefits under different strategy choices and then updates its strategy according to aspiration-driven dynamics. The above process is repeated until the proportion of a certain strategy tends to be stable in the whole population. Obviously, the value of k determines the payoffs—ak and bk. All possible payoffs of a focal individual are uniquely defined by the number of A in the group, and the payoff matrix is as follows.
For any group engaging in a one-shot game, we can obtain each member’s payoff according to Table 2. When X chooses strategy A, the total contribution by individuals to the swarm is kc+c, the total gain of the swarm is rkc+c multiplied by the profit coefficient r, and the gain of each individual is rkc+c/d. As the cost of X is c, the net gain of X is rkc+c/d−c. When X chooses strategy B, the total contribution by individuals to the swarm is kc, the total gain of the swarm is rkc, and the gain of each individual is rkc/d. As there is no cost in such a case, the net gain of X is rkc/d. Thus, the payoffs for A and B are (1)ak=rkc+cd−c,(2)bk=rkcd.
Payoff in multiplayer evolutionary game.
d−1
...
k
...
0
A
ad−1
...
ak
...
a0
B
bd−1
...
bk
...
b0
3.2. Expected Payoff for Strategies A and B
In a finite well-mixed population of size N, groups of size d are assembled randomly, so the probability of choosing a group that consists of another k players of type A and d−k−1 players of type B is given by a hypergeometric distribution [45]. For example, the probability that an A player is in a group of k other A’s is given by PAN,i;d,k=Ci−1kCN−id−k−1/CN−1d−1, where i is the number of A players in the population. The symbol Cnk denotes a combinatorial notation, which is the number of ways to choose a k element subset from an n element set.
The expected payoffs for any A or B in a population of size N, with i players of type A and N−i players of type B, are given by (3)πAi=∑k=0d−1Ci−1kCN−id−k−1CN−1d−1ak,(4)πBi=∑k=0d−1CikCN−i−1d−k−1CN−1d−1bk.
3.3. Aspiration-Driven Dynamics
There are several typical strategy update rules in evolutionary game, such as unconditional imitation [35], Fermi rule [46, 47], etc. Aspiration-driven dynamics focuses on comparing the payoff with aspiration level to make new decisions. Players need not see any particular payoffs but their own, which they compare with an aspiration value. The aspiration-driven dynamics coincides with the requirement of self-management and self-coordination of unmanned swarm in the case of incomplete information acquisition in complex battlefield. The level of aspiration, α, is a variable that influences the stochastic strategy updating. The probability of switching strategy is random when individuals’ payoffs are close to the level of α, reflecting the basic degree of uncertainty in the population. When payoff exceeds α, strategy switching is unlikely. At high values of α compared with payoff, switching probabilities are high.
To model stochastic aspiration-driven switching (from strategy A to B), we can use the following probability function: (5)PA⟶B=11+eωπAi−α.
The aspiration level, α, provides the benchmark used to evaluate how “greedy” an individual is. Higher aspiration levels mean that individuals aspire to higher payoffs. The intensity of selection, ω, provides a measure of how important individuals deem the impact of the actual game on their update. Let Δ=πA−α; if Δ=0, then PA⟶B=1/2, which means that individuals have the same preference for strategies A and B; if Δ>0 (i.e., the individual payoff πA is higher than aspiration level α), then PA⟶B<1/2, which means individuals prefer strategy A; if Δ<0 (i.e., individual payoff πA is lower than aspiration level α), then PA⟶B>1/2, which means individuals prefer strategy B. As for whether an individual updates strategy or keeps strategy unchanged in a certain round of game, it can be further determined by other algorithms, such as roulette algorithm.
In the same way, the probability of the focal individual updating from strategy B to A is (6)PB⟶A=11+eωπBi−α.
In the aspiration-driven dynamics, at each time step, the number of strategy A, i.e., i, can only increase by one, decrease by one, or stay the same. When the number of strategy A increases by one, two subsequent events happen: first, a B strategy individual is selected from the population; then it does not satisfy with the payoff it obtains and switches to the strategy A. A similar process holds for the number of strategy B. Therefore, the probability that the number of A individuals changes at one time step is(7)Pi⟶i−1=Ti−=iN11+eωπAi−α,(8)Pi⟶i+1=Ti+=N−iN11+eωπBi−α,(9)Pi⟶i=1−Ti−−Ti+.
Because there is a stable distribution in the Markov chain without absorbing state, the average abundance function of multiplayer evolutionary game can be derived based on the above state transformation equation.
4. Average Abundance Function
At present, most of the research on average abundance is based on digital simulation, but no strict mathematical expression is given. In this part, we first give the definition of the average abundance of unmanned swarm and then derive its mathematical expression by analyzing the stable distribution of the nonabsorbent Markov chain to support the subsequent simulation analysis in Section 5.
4.1. Average AbundanceDefinition 1.
(average abundance of unmanned swarm). Set the proportion j/N of unmanned units with strategy A in a swarm as a random variable. Let νj be the probability distribution of j/N; then the expected value of j/N is defined as the average abundance of unmanned units with strategy A.
Therefore, the definition of average abundance XAj can be expressed as (10)XAj=∑j=0NjNvj.
The key to calculating the average abundance is to determine the probability distribution νj. For Markov chains without absorbing state, νj is just the stable distribution φjj∈0,N, and it satisfies the detailed balance condition [48]: φjTj+=φj+1Tj+1−.
Equation (10) is just a definition formula, which cannot be directly applied to the actual calculation and analysis. Next, we will theoretically deduce the average abundance formula based on the detailed balance condition so as to reveal the quantitative relationship between the average abundance and related parameters (cost, multiplication factor, selection intensity, etc.) and provide a theoretical calculation basis for the subsequent characteristic analysis.
4.2. Function Deduction
It can be derived from the detailed balance condition: (11)φ1=T0+T1−φ0,(12)φ2=T1+T2−φ1=T0+T1+T1−T2−φ0,(13)φ3=T2+T3−φ2=T0+T1+T2+T1−T2−T3−φ0.
Further, we induce and summarize the above formulas; then we get (14)φj=∏i=0j−1Ti+∏i=1jTi−φ0=∏i=0j−1hiφ0j≥1,where hi=Ti+/Ti+1− is the strategy dominant function. If hi>1, that is, the increasing probability of strategy A is greater than the decreasing probability, it means that strategy A is dominant in the swarm; otherwise, strategy B is dominant.
Since ∑j=0Nφj=1, (15)∑j=0Nφj=φ0+∑j=1N∏i=0j−1hiφ0=1.
Then, we have φ0=1/1+∑j=1N∏i=0j−1hi. Inserting φ0 into formula (14), we have (16)φj=∏i=0j−1hi1+∑j=1N∏i=0j−1hij≥1.
Inserting (16) into (10), XAj can be written as (17)XAj=1N∑j=1Nj∏i=0j−1hi1+∑j=1N∏i=0j−1hi=1N∑j=1Nj∏i=0j−1hi1+∑j=1N∏i=0j−1hi,(18)hi=Ti+Ti+1−=N−i1+eωπAi+1−αi+11+eωπBi−α.
Equations (17) and (18) are just a general expression for the average abundance of multiplayer evolutionary games under aspiration-driven dynamics, and the specific application depends on ak and bk. Therefore, the combination of equations (1)–(4), (17), and (18) forms the average abundance function of unmanned swarm under the framework of multiplayer public goods evolutionary game.
5. Evolutionary Game Analysis
On the basis of the average abundance of unmanned swarm obtained above, we will analyze the impact of cost c, multiplication factor r, and aspiration level α on it. Set the basic parameters N=100, d=15, c=1, r=1.3, and α=1, and when calculating the impact of one parameter, others remain unchanged. In addition, in order to highlight the different influence degree of parameters on average abundance under different selection intensities, ω=0,5,10,15,20 is selected in each simulation scenario.
5.1. Average Abundance With Respect to Cost
It can be easily proved through mathematical induction from equations (1) and (2) that increasing c will increase ak and bk and then increase πAi and πBi, resulting in the decrease of both Ti+ and Ti−. Since hi=Ti+/Ti+1−, in the case of increasing c, the change of hi and XA are difficult to directly determined. Next, we will give a set of numerical solutions to intuitively observe the interaction within a certain range, so as to reveal the influence of c on XA through simulation.
Select the interval c∈0.9,1.8 and draw the average abundance curve of the strategy A as follows.
As shown in Figure 2, as c increases, XA will monotonically decrease; when ω=0, XA=0.5 (i.e., the proportion of collaborators and betrayers in the swarm is balanced), while ω≠0, XA increases with ω; moreover, with the decrease of ω, the influence degree of c on XA increases: ΔXAω=20≈0.028, while ΔXAω=5≈0.063.
Average abundance with respect to cost.
5.1.1. Conclusion 1
The increasing of cost will decrease the average abundance, especially when selection intensity is small.
5.2. Average Abundance with Respect to Multiplication Factor
Similarly, in the case of r increasing, the change of hi and XAcannot be determined only by deduction. Select the interval r∈0.9,1.8 and draw the average abundance curve of the strategy A under different selection intensities as shown in Figure 3.
Average abundance with respect to multiplication factor.
As the multiplication factor increases, XA will monotonically decrease, which means the phenomenon of “free riding” appears, resulting in the weakening of cooperation and the decline of XA; moreover, with the decrease of ωω≠0, the influence of r on XA increases: ΔXAω=20≈0.014, while ΔXAω=5≈0.047.
5.2.1. Conclusion 2
The increasing of multiplication factor will decrease the average abundance, especially when selection intensity is small.
5.3. Average Abundance with respect to Aspiration Level
Select the interval α∈0.9,1.4 and draw the average abundance curve of the strategy A under different selection intensities as shown in Figure 4.
Average abundance with respect to aspiration level.
As the aspiration level increases, XA will monotonically increase, which means the rising of aspiration level makes it more difficult for betrayers to satisfy their expectations, and thus more betrayers transfer to cooperators; moreover, with the decrease of ωω≠0, the influence degree of α on XA increases: ΔXAω=20≈0.002, while ΔXAω=5≈0.037.
5.3.1. Conclusion 3
The increasing of aspiration level will increase the average abundance, especially when selection intensity is small.
According to the simulation results, c, r, and α have an impact on the curve trend of average abundance. When c and r increase, the average abundance decreases monotonically, while, with the increase of α, the average abundance increases monotonically. The conclusions from the simulation provide a theoretical basis for the regulation of swarms in practical application. Based on the conclusions above, in the following section, a case study is provided to further reveal the cooperative evolution mechanism of the unmanned swarm.
6. Case Study
Fire strike is a typical task in unmanned swarm operation. Limited by the ammunition loading/mounting capacity, when the unmanned swarm carries out the fire strike task in case of failure of centralized control mode, the “rational” unmanned units with intelligence and decision-making ability will strictly control the ammunition launching/delivery quantity with “free riding” mentality, while from the perspective of the whole swarm combat effectiveness, we hope that each unit can provide as much ammunition as possible to ensure the overall strike effectiveness on enemy. The key to coping with this contradiction is how to raise the proportion of cooperators in the swarm through self-regulation and self-coordination.
Consistent with the above section, setN=100, d=15, c=1, r=1.3, and α=1 and draw the basic curve (see Figure 5). Since XA<0.5, this case is a nondominant case; that is, most units choose strategy B. Therefore, we try to regulate relevant parameters to increase the average abundance of unmanned swarm and promote cooperation.
Effects of different factors on average abundance.
As in Figure 5, reducing the cost or increasing the aspiration level can raise the proportion of cooperators. However, increasing the multiplication factor will cause the average abundance curve to deviate downward from the basic curve, which is because increasing payoff of cooperators and betrayers by the same margin will make the “free riding” situation more serious. Consequently, we try to separate the multiplication factor of cooperators from that of betrayers, only increase the multiplication factor rA of cooperators (the multiplication factor rB of betrayers remains unchanged), and find that the average abundance curve deviates upward from the basic curve.
Furthermore, we simulate the average abundance under different rA (see Figure 6). When rA=2, the average abundance is approximately equal to 0.5, which indicates that the proportion of cooperators and betrayers in the swarm is basically balanced. With the further increase of rA, when rA=2.65, the average abundance will be greater than 0.5 at ω≈10, while when rA=2.9, the average abundance will be greater than 0.5 at ω≈5. Thus, we can reach the following conclusions.
The adjustment on rA can switch the dominant strategy, making the average abundance of strategy A greater than 0.5.
The lower the ω is, the more stringent requirement for rA will be, and the higher the ω is, the looser requirement for rA will be: XAω=5,rA=2.9>0.5, while XAω=10,rA=2.65>0.5.
Effects of different rA on average abundance.
In order to investigate the regulation sensitivity of different parameters, we simulate the affecting degree of unit variation of c, α, and rA on the average abundance. We select the simulation results with ω=0,5,15 to be discussed, as shown in Figures 7(a)–7(c), respectively.
When ω=0, the average abundance is identically equal to 0.5, and thus the parameter regulation loses its effect (see Figure 7(a)).
When ω≠0 and unit variation of parameters (i.e., Δ) is small (note that the threshold of Δ is related to ω: Δ≈1.70|ω=5, Δ≈1.53|ω=15), the change in value of average abundance caused by adjusting α and c is much greater than adjusting rA (see Figures 7(b) and 7(c)). The regulation of α and c is more sensitive than that of rA.
When ω≠0 and Δ is large, the regulation effect of rA is much better than that of α and c. And the larger ω is, the more sensitive rA is; i.e., a small ΔrAleads to a large increasing in average abundance: ΔXAω=5,ΔrA=1.95≈0.43, while ΔXAω=20,ΔrA=1.55≈0.48 (see Figures 7(b) and 7(c)).
Effects of unit variation in parameters on average abundance. (a) ω=0. (b) ω=5. (c) ω=15.
To improve the average abundance, the ideal measure is to increase the multiplication factor, reduce the cost of cooperators, or both. However, in order to ensure the effectiveness of the operation in the actual battlefield, the cost is difficult to reduce or even increase. Therefore, it is necessary to consider increasing both rA and c. Figure 8 shows the change of average abundance when rA and c increase at the same time (c increases by 50%, and rA increases by 69% and 73%, respectively). Accordingly, as long as rA increases by more than 73%, not only can the adverse effect of cost increasing on average abundance be offset, but also the cooperation in swarm can be promoted.
Effect of increase in cost and multiplication factor on average abundance.
Unfortunately, the above regulation can only achieve a limited increase in the average abundance; that is, it cannot make the average abundance greater than 0.5. According to the conclusions from Figures 5 and 6, the conversion of dominant strategies (a large increase in average abundance) depends on a large selection intensity ω and a large unit variation ΔrA, and thus we further increase rA under the premise of increasing c by 50% (see Figure 9). According to the results in Figure 9, when rA=2.52 and ω≈15, XA will be greater than 0.5; when rA=2.65 and ω≈5, XA will be greater than 0.5.
Strategy alternation by increasing cost and multiplication factor.
The increase of rA means that the hitchhiker will no longer get as much payoff as the cooperator, and the decrease of payoff will directly increase the strategy update probability PB⟶A, so then more units tend to cooperate (more betrayers transfer to cooperators).
According to the above simulation results and conclusions, we can consider the following measures from two dimensions of management and technology in the actual control of unmanned swarm cooperation:
Increase the multiplication factor value rA of cooperators as much as possible. For example, with the help of advanced management means, for each combat unit in the swarm, its investment (i.e., cost c) in previous operations can be accumulated, and those with higher cumulative investment will be given more supplies (e.g., ammunition) or higher supply priority in the follow-up operations.
Minimize the cost c for each operation. For example, with the help of advanced technology means, improve the reliability and survivability of combat units or the strike accuracy and damage-power of ammunition.
In addition, since c,r, and α are closely related to specific operation tasks, it is also necessary to discuss specific control measures in combination with operation tasks under the limitation of parameter value range.
7. Conclusion
The advantage of unmanned swarm operation lies in its autonomy that it can continually conduct cooperative operation efficiently in case of combat unit damage or communication failure. This work aims at the autonomous collaboration of the unmanned swarm under the failure of centralized control mode and proposes a cooperative evolution mechanism within the framework of multiplayer public goods evolutionary game. We get the average abundance function by theoretical derivation and then simulate the influence of different parameters (i.e., c, r, and α) on the abundance. The simulation results of unmanned swarm fire attack show that increasing the multiplication factor rA and reducing the cost c can improve the average abundance of cooperators; furthermore, when the unit variation Δ is large, rA not only has a high regulation sensitivity, but also can realize the switching of the dominant strategy. Finally, we suggest some proposals to provide an exploration for the transformation from theory to application.
The evolution of cooperation is a fascinating topic that has been studied from different perspectives and theoretical approaches. Our approach by means of multiplayer public goods evolutionary game sheds new light on how to study and analyze the evolution of cooperation in the unmanned swarm. In our work, we assume that the units in the swarm are homogeneous, which indicates a globally consistent α in the process of strategy updating. However, in reality, different units (firepower units, intelligence units, etc.) probably have various requirements for α. Thus, how to get the average abundance and explore the cooperative evolution mechanism when multiple α coexist will be our further work.
Data Availability
The data used to support the findings of this study are available from the corresponding author upon request.
Conflicts of Interest
The authors declare that they have no conflicts of interest regarding the publication of this paper.
Acknowledgments
This work was supported by the National Natural Science Foundation of China (no. 71901217); the National Key R&D Program of China (no. 2018YFC0806900); China Postdoctoral Science Foundation funded Project (no. 2018M633757); and the Primary Research and Development Plan of Jiangsu Province under Grant no. BE2019762.
XuP.DherbomezG.HeryE.AbidliA.BonnifaitP.System architecture of a driverless electric car in the grand cooperative driving challenge2018101475910.1109/mits.2017.27761352-s2.0-85041018095MaY.HuM.YanX.Multi-objective path planning for unmanned surface vehicle with currents effects20187513715610.1016/j.isatra.2018.02.0032-s2.0-85042087186ShakeriR.Al-GaradiM. A.BadawyA.Design challenges of multi-UAV systems in cyber-physical applications: a comprehensive survey, and future directions20199910.1109/COMST.2019.293FanJ.LiD.LiR.Analysis on MAV/UAV cooperative combat based on complex network201916115015710.1016/j.dt.2019.09.0022-s2.0-85072621787QuanJ.ChuY. Q.WangX. J.Public goods with punishment and the evolution of cooperation2019391141149SmithJ. M.PriceG. R.The logic of animal conflict19732465427151810.1038/246015a02-s2.0-34548719708NowakM. A.SigmundK.Evolution of indirect reciprocity200543766851291129810.1038/nature041312-s2.0-27644522801LiebermanE.HauertC.NowakM. A.Evolutionary dynamics on graphs2005433702331231610.1038/nature032042-s2.0-12844260210NowakM. A.Five rules for the evolution of cooperation200631458051560156310.1126/science.11337552-s2.0-33845415805WolefI.What are the equilibria in public-good experiments?20171508385NowakM. A.TarnitaC. E.AntalT.Evolutionary dynamics in structured populations20103651537193010.1098/rstb.2009.02152-s2.0-75449114445PenaJ.WuB.ArranzJ.Evolutionary games of multiplayer cooperation on graphs201612810.1371/journal.pcbi.10050592-s2.0-84988928502SuQ.McAvoyA.WangL.Evolutionary dynamics with stochastic game transitions2019https://arxiv.org/abs/1905.10269HindersinL.WuB.TraulsenA.Computation and simulation of evolutionary game dynamics in finite populations201991694610.1038/s41598-019-43102-z2-s2.0-85065330910OhtsukiH.HauertC.LiebermanE.NowakM. A.A simple rule for the evolution of cooperation on graphs and social networks2006441709250250510.1038/nature046052-s2.0-33745317468FuF.NowakM. A.HauertC.Invasion and expansion of cooperators in lattice populations: prisoner’s dilemma vs. snowdrift games2010266335836610.1016/j.jtbi.2010.06.0422-s2.0-77954970834MaciejewskiW.FuF.HauertC.Evolutionary game dynamics in populations with heterogenous structures201410410.1371/journal.pcbi.10035672-s2.0-84901363409FotouhiB.MomeniN.AllenB.NowakM. A.Evolution of cooperation on large networks with community structure2019161522018067710.1098/rsif.2018.06772-s2.0-85062891113JosefT.AndreasP.KrishnenduC.Population structure determines the tradeoff between fixation probability and fixation time20192AllenB.LippnerG.NowakM. A.Evolutionary games on isothermal graphs20191011910.1038/s41467-019-13006-7MartaD. S.FlávioL. P.FranciscoC. S.Dynamics of N-person snowdrift games in structured populations20123158186LiP. P.KeJ.LinZ.Cooperative behavior in evolutionary snowdrift games with the unconditional imitation rule on regular lattices20128522111110.1103/physreve.85.0211112-s2.0-84857615248WangX. J.XiaK.Extended average abundance function of multi-player snowdrift evolutionary game under aspiration driven rule201939511281137TarnitaC. E.OhtsukiH.AntalT.FuF.NowakM. A.Strategy selection in structured populations2009259357058110.1016/j.jtbi.2009.03.0352-s2.0-67649979847DuJ. M.WuB.WangL.Aspiration dynamics in structured population acts as if in a well-mixed one[J]20155801410.1038/srep080142-s2.0-84923055356YuM.HeM.ZhangD.An approach to coordinated control of structured unmanned swarm based on evolutionary gameProceedings of the 2020 3rd IEEE International Conference on Unmanned SystemsNovember 2020Harbin, ChinaYuM.ZhangD.KangK.Cooperative evolution mechanism of unmanned cluster based on multi-player public goods evolutionary game2020421227872794LiuQ.HeM.LiuJ.A mechanism for identifying and suppressing the emergent flocking behaviors of UAV swarms2019472374381GuL.HeM.ChenG.Research on unmanned aerial vehicle swarm system resilience202043115616210.3969/j.issn.1001-506X.2021.01.19LiuQ.HeM.Research on the requirements and countermeasures of counter equipment for small UAV swarm at sea20193345965SuiX.CongR.LiK.Evolutionary dynamics of N-person snowdrift game201537945-462922293410.1016/j.physleta.2015.08.0292-s2.0-84945491463PeñaJ.WuB.TraulsenA.Ordering structured populations in multiplayer cooperation games2016131142015088110.1098/rsif.2015.08812-s2.0-84958597724WuB.TraulsenA.GokhaleC.Dynamic properties of evolutionary multi-player games in finite populations20134218219910.3390/g40201822-s2.0-84880389178DuJ.WuB.AltrockP. M.WangL.Aspiration dynamics of multi-player games in finite populations201411942014007710.1098/rsif.2014.00772-s2.0-84896968674DuJ.WuB.WangL.Aspiration dynamics and the sustainability of resources in the public goods dilemma2016380161432143610.1016/j.physleta.2016.02.0392-s2.0-84959419479DuJ. M.WuB.WangL.Evolutionary game dynamics of multi-agent cooperation driven by self-learningProceedings of the 2013 9th Asian Control Conference (ASCC)June 2013Istanbul, TurkeyIEEE16WuB.AltrockP. M.WangL.Universality of weak selection20108244610610.1103/physreve.82.0461062-s2.0-78651277890XuK.LiK.CongR.WangL.Cooperation guided by the coexistence of imitation dynamics and aspiration dynamics in structured populations201711744800210.1209/0295-5075/117/480022-s2.0-85018525025WangX. J.GuC. L.LvS. J.Evolutionary game dynamics of combining the Moran and imitation processes201928211310.1088/1674-1056/28/2/0202032-s2.0-85062567215ChenY.-S.YangH.-X.GuoW.-Z.Aspiration-induced dormancy promotes cooperation in the spatial prisoner’s dilemma games201746962563010.1016/j.physa.2016.11.0822-s2.0-85000352366PlatkowskiT.Aspiration-based full cooperation in finite systems of players20152514654WangX. J.LiuJ.Stable coalition structures in cooperative game with externalities201838511731183LiuX.HeM.KangY.PanQ.Fixation of strategies with the Moran and Fermi processes in evolutionary games201748433634410.1016/j.physa.2017.04.1542-s2.0-85019898313LiuX.HeM.KangY.Aspiration promotes cooperation in the prisoner’s dilemma game with the imitation rule20169411212410.1103/physreve.94.0121242-s2.0-84979207910GrahamR. L.KnuthD. E.PatashnikO.19942ndBoston, MA, USAAddison-Wesley Publishing CompanySzabóG.TőkeC.Evolutionary prisoner’s dilemma game on a square lattice1998581697310.1103/physreve.58.692-s2.0-0002180201NowakM. A.SigmundK.Evolutionary dynamics of biological games2004303565979379910.1126/science.10934112-s2.0-0842288340Van KampenN. G.1992Amsterdam, NetherlandsElsevier