In multiagent systems (MASs), agents need to forward packets to each other to accomplish a target task. In this paper, we study packet forwarding among agents using evolutionary game theory under the mechanisms of Carrier Sense Multiple Access/Collision Avoidance (CSMA/CA). Packet forwarding among agents plays a key role to stabilize the whole MAS. We study the transfer probability of packet forwarding of agents at the idle state or the busy state and computer the probability of the packet forwarding for a MAS. When agents make their decisions to select
Multiagent systems (MASs) are computerized systems composed of multiple interacting intelligent agents that have limited energy and selforganization ability. To achieve a common goal, MASs require them to selforganize into a network and collaborate with one another [
The agents need to interact with each other as much as possible to accomplish a target task. Cooperation is a fundamental problem in the distributed control community; the agents must effectively cooperate with each other for mutual benefit [
As a mathematical tool, game theory mainly focuses on the competitive and cooperative relationship of the participants, and it has been widely applied in the field of cooperation. Typical examples of evolutionary games include repeated prisoners’ dilemma and snowdrift games [
In the paper, using a Markov chain model, we calculate the probability of packet forwarding while agents are in idle or busy states and analyze the packet forwarding behaviors of the agents. We then build a packet forwarding strategies game model, which defines the payoffs of the different strategies. Moreover, we introduce two incentive mechanisms into the game, one to increase agent cooperation by selecting
(1) The Markov chain model is used to calculate the probability of packet forwarding for an agent at the idle or busy state. We establish a packet forwarding strategies game model with two incentive mechanisms. One incentive mechanism is to promote cooperation among agents by forwarding packets and the other is to promote noncooperation among agents by selecting the
(2) Replicator dynamics is used to prove the evolutionary stability theorems and inferences, which provide the conditions to attain stable states for the packet forwarding strategies game model in MASs.
(3) The experiments verify the correctness of the provided theorems and inferences. Also, the experiments show the effects of two incentive mechanisms and the probability of packet forwarding of agents on the rate of convergence to stable states.
The rest of the paper is organized as follows. We first discuss the related research in Section
In recent years, to provide an incentive for the agents/nodes to cooperate, two mechanisms have been devised: incentive mechanisms and punishment mechanisms, which incentivize agents/nodes to forward the others’ packets and punish misbehaving nodes. Most research involves strengthening packet forwarding among nodes/agents by setting up incentivebased systems [
Also packet forwarding strategies and cooperative strategies among nodes have been studied by different types of game theory in wireless networks, such as repeated games [
Zhu et al. [
Among these methods of evolutionary game theory, some stimulation mechanisms are proposed to incentivize the nodes’ cooperation. Shen et al. [
Among these methods of evolutionary game theory, some researchers study packet forwarding among nodes in different scenarios. Li et al. [
Compared to [
Packet forwarding among agents in MASs is not a deterministic process. The present value of the process is independent of all past values. The process is a noaftereffects process and hence has Markov properties.
Time series of packet forwarding can be expressed by a Markov chain. If
In this section, we build a Markov model of packet forwarding among agents in MASs. Let us assume the following:
(1) If an agent has a new packet to forward, it forwards it at the beginning of the next timeslot.
(2) If an agent successfully forwards its packet, it can forward a new packet in the next timeslot.
(3) If an agent detects a collision, it forwards its packet in each subsequent timeslot until the packet is successfully forwarded.
With the assumptions, each agent is in either an idle state or a busy (backlogged) state. An agent is in the busy state if the number of packets that need to be transmitted exceeds its transmitting capacity, which leads to buffer overflow and loss of data packets. Furthermore, because agents have limited computing and communication capability with a multihop and manytoone communication method, when the wireless communication channels present noise, or the network topology changes, or the packets suddenly increased because of an emergency, the agents may easily be in a busy state that will cause network delay and loss of data packets. The decision to forward the packet or not depends only on whether the agent is busy or not. Therefore, the packet forwarding decision in one timeslot for each agent is Markovian. Since agents always have packets to forward, the notations are shown in Table
Notations (1).

The number of agents in the MASs. 



The idle and busy states of an agent, respectively. 



The probabilities of packet forwarding of an agent at the idle state and the busy state, respectively. 



The probability of packet forwarding at idle state of an agent. The probability of packet no forwarding at the idle state of an agent is 



The probability of packet no forwarding at the busy (backlogged) state of an agent. The probability of packet forwarding at busy state of an agent is 



The state transition matrix of the Markov chain of an agent. 
Packet forwarding is regarded as a twostate Markov chain in MASs. The state transition diagram of the Markov chain is shown in Figure
The state transition diagram of the Markov chain of an agent.
The state diagram of a Markov chain describes the probability of all state transitions and their probabilities and is expressed in matrix form as follows [
The issue is how to determine the probabilities of the other timeslots once the probability of the initial state is given.
Suppose the initial probabilities of packet forwarding at the idle state and the busy state are
The initial state is expressed by the initial probability row vector, that is,
The probability of the next timeslot is
As time progresses from n to n+1, the general expression is
Another way to write these equations is
Now, let
Notice that
A multiagent system is a system that can accomplish a common goal with a number of mutually independent agents which possess the ability of selforganization, learning, and reasoning. It is critical that all agents have the independent ability to cooperate with each other, which ensures accomplishing the expected task for a multiagent system.
When agents perform a task in a noisy environment, they may be disturbed by some external factors. The main reason for loss packet is network congestion during the packet forwarding process. Assume that all agents have the same communication status factors in the MASs and that all agents have the same probability of successfully forwarding and receiving packets at any one time.
Communication among agents depends on packet forwarding with each other, and each agent has two strategies to select:
Notations (2).

Gain obtained by an agent selecting 



The incentive for noncooperation, gain obtained by an agent selecting 



Cooperation gain of an agent due to its opponent selecting the 



Cost caused by an agent forwarding others’ a packet or sending its own packet. 



No cooperative loss of an agent for its opponent selecting 



The incentive for cooperation while an agent selects the 
When agents interact with each other in MASs, each agent can make a decision to select the strategy either
When two agents A and B transmit packets, each has two strategies:
Twoagent packet forwarding model.
There are three situations of packet forwarding strategy between two agents: both interactive agents select
An agent might fail to transport packets due to the channel noise in MASs. Each packet forwarding strategy results in either success or failure. Suppose two agents, A and B, select one strategy. There are 4 different events in total. Table
Payoffs of two interactive agents selecting the
Serial number  A  B  Probability  A profit G(A)  B profit G(B) 
Forward  Forward 
 


1  1  1 





2  1  0 





3  0  1 





4  0  0 



In Tables
Payoffs of only one agent selecting the
Serial number  A  B  Probability  A profit G(A)  B profit G(B) 
Forward  NoForward 
 


1  1  1 





2  1  0 





3  0  1 





4  0  0 



Payoffs of two agents both selecting the
Serial number  A  B  Probability  A profit G(A)  B profit G(B) 

NoForward  NoForward 
 
1  1  1 





2  1  0 





3  0  1 





4  0  0 



In Table
where
When two agents A and B interact, agent A selects the
In Table
and the payoff of agent B selecting the
where
Two agents A and B select the
In Table
The packet forwarding game in MASs is a quadruple
(1)
(2)
(3)
(4)
Payoff matrix.
Forward  NoForward  

Forward 




NoForward 


The dynamic evolutionary process can be described by many different dynamic replicator models. There is a very widely used and famous dynamic replicator model proposed by Taylor and Jonker [
where
Equation (
We have two strategies in the packet forwarding game model; we denote
If
If
If
Mathematically, the stability theory of the differential equations implies that if
Because
Taking the derivative on both sides of (
According to the stability theory of the differential equation,
When
When
Suppose
From (
and due to (
If
If
Under the conditions of Theorem
Under the conditions of Theorem
Under the conditions of Theorem
Under the conditions of Theorem
Under the conditions of Inference
From Theorem
In the case that an agent’s communication environment is backlogged and the agent selects
From (
The positive changing relations among
There are many ways to improve the probability of packet forwarding in MASs, such as increasing bandwidth or frequency, improving the hardware or the transmission protocol algorithm, etc. From Theorem
The experimental environment is MATLAB 2015a. The simulation experiments are divided into two parts.
(1) The experiments in the first part are to verify the correctness of Theorems
(2) The experiments in the second part are to confirm the effects of parameters
We set parameters to meet the conditions of Theorems
Experimental data (1).
Group number  Satisfy 







Theorem  

Theorem 
0.8  12.5  5  7  3  4 

Theorem 
0.8  12.75  5  7  3  2.6 

Theorem 
0.8  15  5  7  3  2 

Theorem 
0.8  15  5  7  3  4 

Theorem 
0.8  5  1  3  2  1 
The evolution curves of the packet forwarding game (1).
Evolution curves of the packet forwarding game (2).
Evolution curves of the packet forwarding game (3).
Evolution curves of the packet forwarding game (4).
In Figure
In Figure
In Figure
In Figure
We set many groups of parameters to meet the conditions of related Theorems. Groups 16 vary the value of
Experimental data (2).
Group number  Satisfy 



C  L  W 

Theorem  

Theorem 
0.8  12.5  5  7  3  3 

Theorem 
0.8  12.5  5  7  3  4 

Theorem 
0.8  18  3  8  4  3 

Theorem 
0.8  18  3  8  4  4 

Theorem 
0.8  9  4  8  4  3 

Theorem 
0.8  9  4  8  4  4 

Theorem 
0.8  12.5  6  7  3  4 

Theorem 
0.8  12.5  5  7  3  4 

Theorem 
0.8  15  3  8  4  4 

Theorem 
0.8  15  4  8  4  4 

Theorem 
0.8  8  8  8  4  4 

Theorem 
0.8  8  3  8  4  4 

Theorem 
  18  3  8  4  3 

Theorem 
  9  0.8  4  4  5 
In Figures
Evolution curves of the packet forwarding in terms of
Evolution curves of the packet forwarding in terms of
Evolution curves of packet forwarding in terms of
In Figure
In Figure
In Figure
In Figures
Evolution curves of the packet forwarding in terms of
Evolution curves of the packet forwarding in terms of
Evolution curves of the packet forwarding in terms of
In Figure
In Figure
In Figure
From Figures
In Figures
Evolution curves of the packet forwarding in terms of
Evolution curve of the packet forwarding in terms of
In Figure
In Figure
In Figure
Evolution curves of the packet forwarding in terms of P with no constant
The packet forwarding mechanism is an important aspect of intelligent task allocation and collaborative work research in MASs. In this paper, based on evolutionary game theory, we have studied packet forwarding strategy decisions and the evolutionary process. Considering the real network communication environment, we have explored the probability of packet forwarding while the agent is idle or busy (backlogged). While the agent is in a busy state, we have introduced a noncooperation incentive to encourage agents to select the
We have established rules of packet forwarding decisionmaking in the process of dynamics evolution in MASs. In subsequent work, we can further study the relation between the value of noncooperation incentive of agents and the probability of agents in idle state. The packet forwarding strategy game model proposed provides a method of network safety.
The data used to support the findings of this study are available from the corresponding author upon request.
The authors declare that there are no conflicts of interest regarding the publication of this paper.
This work was supported by Studying Abroad Fund of Shanghai Municipal Education Commission. This work was advised by Professor Wynn Stirling of Brigham Young University and polished by our friend Chad Josephson. The authors express their hearty thanks for their help here.