Emergence of Collective Behavior in a System of Autonomous Agents

We study the properties of emergent collective behavior in a system of interacting autonomous agents. The system evolves in time according to two rules of communication, one from interactions between agents and another one from interactions ofthe agents with an additional agent which evolves in time in a periodic and stable manner. The application of these two rules is decided by a prescribed probability distribution.We analyze the emergence and the efficiency (coordination) with which collective patterns are constructed in time as a function of the parameters of the system.


INTRODUCTION
These days, the term agent (as in multi-agent systems) and its derivatives acquires different meaning in various areas of the fast growing sciences and technologies of information.In this work we employ the general term agent as a mobile independent entity, able to communicate and interact with its environment and other agents.This term is well in accordance with the term's definition, quite widely accepted as in [12].Agent can be used in models for biological [1] and/or robot communities [6].We conjecture that these kinds of modeling capture some elementary aspects of the dynamics of social self-organizing systems, like the spread of rumors (or information in general) in intercommunicating systems.
Since now the study of the collective behavior of autonomous agents has been performed using Monte Carlo methods [4].In [4] each agent is programmed with the same rules of movement, based on the signals received from other agents.Each agent emits an omni-directional signal and in turn receives external signals using four receptors (ahead, behind, left, right) capable to identify at most four directions in space.Its position and movement affect the position and movement of other nearby agents and vice-versa.While no group movement is programmed, patterns regularly emerge from such interactions.
In [5] it is considered a set of basic interactions or primitives, which can be used to structure and simplify the process of designing and analyzing group behaviors.There the main goal is to intro- duce the types of simple local interactions which can produce complex and purposive group behaviors.It was demonstrated how these basic interactions can be simply combined into more complex compound group movements.The developed set of inter- actions was tested on a herd of physical mobile robots demonstrating the existence of patterns of collective behaviors: avoiding, following, dispersing, aggregating, homing, flocking, and herding.This investigation has shown that for simple spatial behaviors no dominance hierarchies are necessary, or indeed helpful.
In this work we study the behavior of a group of autonomous agents by means of a cellular auto- maton.Cellular automata arose from numerous attempts to create a simple mathematical model describing complex biological structures and pro- cesses [11].A cellular automaton is a simple discrete dynamical system, the behavior of which is totally dependent upon the local interconnections between its elementary parts [7].This kind of modeling turned out to be very productive and has been widely and successfully applied in various fields.From the mathematical point of view cellular automata can be viewed as finite-state machines with feedback.See for example [2,8] and references therein.
Motivated by the early works of Von Neumann [10] and Burks [3], we are investigating the imple- mentation of Probabilistic Cellular Automata in the study of coupled multi-species agent groups.
As a first step we consider the communication between two species governed by a probabilistic rather than a deterministic process.In this way we implement the kind of coupling suggested in [9] following the spirit of probabilistic control for unstable systems.Here the controller (in the language of [9]) is considered to be the population of agents following a specific stable pattern, and as the unstable system the population which tends to move in a dispersive way in which collective patterns may exist but are not stable in time.
We consider the system of autonomous agents divided in two interacting subgroups coupled according to a prescribed probability distribution.
As a result of coupling the whole system behaves collectively in a way which depends upon the values of the free parameters of the system.
In Section 2 we describe the model, the quantities used for analysis and we give a brief description of the computation environment used.Section 3 contains the results and analysis of the collective behavior of the system in relation with its different free parameters.In Section 4 we present the construction of a "path finder" where the group of agents is used for the purpose of pattern recogni- tion.In Section 5 we present concluding remarks.

DESCRIPTION OF THE MODEL
We consider a system composed of 9 agents (designated as ordinary agents) plus one agent denominated the leader agent.The 9 + agents move in a domain of two spatial dimensions.The domain is a square discretized in 300 x 300 square cells.Each cell may be occupied by at least one agent.At equally spaced discrete times each agent may detect the presence of other ordinary agents and/or the leader if they are inside a defined perception region.This region is defined as a domain around each ordinary agent and is divided in four identical angular sectors: ahead, left, right, back.To each sector it is associated a radius of per- ception Ra, R1, Rr, Rb, respectively, for the com- munication between ordinary agents.For the communication between an ordinary agent and the leader agent we define an independent radius of perception RL, equal for the ahead, left, and right directions, see Fig. 1.In all the cases studied here it is considered Rb 0 and RL 0 for the back sector perception, so that all the agents are "blind" with perception sector of communication with the leader

FIGURE
The sectors of perception ahead, left, right (with radius Ra, R1, Rr) for the communication between agents, and for the communication with the leader (with radius RI).respect to this sector.This distinction in four perception sectors, together with the rules of the cellular automaton described below, models the agents with a capability of distinction of some directions in space.
The leader agent is programmed to perform a stable circular path of radius pL, located at the center of the two-dimensional square domain.The leader does not receive any signal but sends its signal to the other 9 agents of the system.
The time evolution is realized by a cellular automaton.As initial condition the ordinary agents are distributed randomly on the cells of the square domain.According to a prescribed probability distribution pL each ordinary agent updates its position in space by interacting with the leader or with another ordinary agent (provided they are perceived in the perception region).If pi =0 the leader agent is considered as an ordinary agent with respect to the updating rules of the cellular auto- maton described below.
All the ordinary agents move with velocity equal to one in modulo.The leader agent moves with a velocity which intensity v may be any positive value but constant in time.In this work we study the cases when vi 0.2, 0.4, 0.6, 0.8, 1.0.The motiva- tion for this choice will be clear in the next section.
Due to the discretization of the domain, a single spatial displacement of the agents means a transi- tion from one cell to its neighbor cell.As the agents are blind with respect to the back sector of perception and as each cell has 8 neighbor cells then the velocity vector of an ordinary agent may have 7 different directions 0=-3r/4,-r/2,-r/4, 0, 7r/4, 7r/2, 37r/4, with respect to the ahead direction.
We notice that the parameter p can be viewed as a measure of noise in the communication channel between the leader agent and the group of ordinary agents.The velocity vL may be interpreted as an anticipated response of the ordinary agents with respect to the leader when vi < 1, or as a measure of similarity between the leader and the ordinary agents, as far as the reaction in time is concerned.
The two radius R, RI measures the range of the agent/agent and agent/leader interactions.
The robustness of collective behaviors of the system depends strongly on R, R and p, and the geometrical characteristics of the collective pattern constructed in space depend on the parameter vi.
The dynamical law is implemented by the set of updating rules of the cellular automaton as follows: 1.According to the prescribed probability pL a probabilistic game is played.If the outcome is (yes) go to rule 2. If the outcome is (not) go to rule 3. 2. Check if the distance to the leader agent is less than RI.Ifyes then follow the leader moving one cell in its direction.If the distance to the leader is larger than RI then go to rule 3.

Find the agents (including eventually the leader)
which are at a distance less than R. If no agents are detected then move ahead.If there is an agent in the ahead sector then move in the ahead direction.If there is an agent in the left sector and no other agent in the ahead sector then move to the left direction.If there is an agent in the right sector and no other one in the ahead and in the left sectors then move to the right direction.
As the domain is a square lattice of cells then in rule 3 the angular displacement left (right) may be of 90 (-90), 45 (-45) or 135 (-135) with respect to the ahead direction.Due to that we introduce a "thermal" like noise according to which when rule 3 applies for a left (right) shift the angular displacement may be any of the three deviations to left (right) or even to ahead which corresponds to an eventual "mistake".The level of such noise determines statistically the mean value of the angular displacement (equal for all ordinary agents) with which the agents shift their direction of movement.
We use reflexive boundary conditions for the interaction between the ordinary agents and the boundaries of the square domain, so that the incoming direction is identical to the outgoing direction with respect to the direction orthogonal to the boundary.
For the analysis the program was realized in three versions: in Pascal for IBM PC/AT under MS DOS, in Fortran for UNIX system and in Java for WWW.

COLLECTIVE BEHAVIOR AND ANALYSIS OF RESULTS
3.1 A Quantitative Measure of

Collective Behavior
In order to study the properties of the collective movements of the system we consider the velocity- velocity correlation function for an ordinary agent.
The correlation function C(-) fully reveals the existence of coherent movement of the group of agents even if individually the agents do not execute a periodic movement.In fact the coherent collective movements that emerge are executed by the group of agents as a whole.If by some reason an agent escapes from the collective pattern of movement another agent outside the pattern is eventually captured by the group executing this movement.The efficiency with which this substitution mechanism is implemented by the system we call the degree of coordination associated with the collective movement.If this collective movement is less coordinated it implies the substitution mechanism is not well performed.This is reflected by an in- creased amount of noise in the velocity correlation function.The Fourier transform of the function C(-), the power spectrum, determines the charac- teristic frequencies of the collective movement.
From the basic individual communications there are three kinds of interactions that take place in time: between agents at the individual level, be- tween agents and the boundaries of the domain, and between the leader agent and the group (or a sub- group) of ordinary agents.The last one is respon- sible for the eventual emergence of collective http://solvayins.ulb.ac.be/fixed/Agents.html.
behavior, and together with the interaction between agents is also responsible for the large or small efficiency (coordination) of the collective movement.
We consider some typical cases in order to stress the important features of the model: the velocity vL 0.2, 0.4, 0.6, 0.8, the radius RL 25, 50, 75,   200, the radius R---50, 200, and the probability pi=0.0,0.1,0.2,..., 1.0.For each simulation it was considered a duration of 20 h (in units of the cellular automaton) which corresponds to 8192 points for the velocity-velocity correlation func- tion.An additional case is also considered where the radius for the ahead, left, and right sectors of perception are different: Ra=30, Rl=Rr 100, vI =0.8.
In the special case RI-0 the agents are insensi- tive to the presence of the leader.Then for different values of R the group of agents may self-organize in different ways.Circular movements may emerge and disappear in time if R is small enough.In the case R 200 and R 0, the agents communicate in long distances and they move as a close group in a linear trajectory changing direction only after col- liding with the boundaries.If RI-0 this grouped movement is disturbed and depending upon the values of vL and pi a different collective pattern is constructed.In this case the leader acts as a perturbation which may contribute for the emer- gence of a new pattern or may destroy any pos- sibility of collective movement.

Dependence with Respect to v
The simplest case is v 1.0.It corresponds to the intuitive idea that the most the agents receive information from the leader, the most the leader is followed by the agents for any value of R = 0 and R =/: 0. As there is no delay in the velocity of the leader with respect to the velocity of the agents the loss of information from the leader comes only from the noise in the communication represented by the probability p.The mean number of agents following the leader decreases uniformly with pi.For p 1.0 all the agents follow the leader in a stable circular movement with period equals to the size of the leader's path.In this case the power spectrum of the velocity-velocity correlation func- tion has a pronounced peak at the frequency f-33 which corresponds to the period T 251, the size of the leader's path.The amplitude of this peak is a measure of the efficiency or coordination of this simple collective behavior.The peak achieves its maximum value for p= 1.0, meaning that the coordination is best performed for this value ofp.
For vL < the situation changes significantly.
With respect to the agents the ordered signal sent by the leader now corresponds to the period T= 2rp/vL, where p is the radius of the leader's path.
The new collective behavior that emerges is a result of the attempt of the group of agents to adapt the collective movement to the new periodic informa- tion of period T. The solution to this new situation is to increase the total length of the collective movement to the new value of T by means of an additional loop internal to the leader's path, see Fig. 2.
At the local individual level this internal loop may be realized when an agent crosses the leader due to their different velocities.At this moment the individual agent perceives the leader at its left, right or back perception sector.If the leader is in its back sector then the agent will continue to move ahead and will quit the collective pattern.On the other hand if the leader is perceived in its left (right) sector then the agent will move left (right).It continues to shift left (right) until the moment it completes the loop and it perceives the leader or other agents agent Internal loop FIGURE 2 An illustration of the loop internal to the leader's path performed by an ordinary agent.again in its ahead perception sector.It is important to note that after eventually escaping the collec- tive pattern the agents arrive back to the leader's neighborhood after some time perceiving the leader in its left perception sector.Therefore loops internal to the leader's path are more often observed than loops external to it.
The collective movement has now a period T= 27rpL/VL.This is illustrated in Figs. 3 and 4, where it is shown the velocity-velocity correlation function and its Fourier transform for vL 0.8.The frequency of the main peak is f= 26 corresponding to T--27rpL/v 314.In Fig. 5, it is shown the power spectrum for v 0.6.The frequency f--20 corresponds to T 419.The collective movement is formed by the leader's path plus an internal loop larger than in the case where v-0.8.
As vi decreases the period T increases.The collective pattern is the leader's circular path plus the internal loop which increases in size inversely with vi.The limiting situation corresponds to v 0.5.In this case the internal loop overlaps with the leader's path, and if vL < 0.5 the loop becomes larger than the main part of the circular path.Obviously this situation cannot be sustained.After crossing the leader the agents continue to move ahead, the loop is not completed and the collective pattern is destroyed. 3.3 Dependence with Respect to p For 0.5 < vi < 1.0 the collective movement has the recursive time T-27rp/v..The amplitude of the peak corresponding to the period T in the power spectrum of the correlation function measures the coordination or efficiency with which the collective movement is performed.This amplitude depends crucially on the probability p which is a measure of the noise level in the communication between the agents and the leader.FIGURE 5 The power spectrum of the velocity correla- tion function for vi.=0.6.The frequency of the main peak decreases as v decreases.
We observe that in this case the maximum value of the amplitude of the main peak of the power spectrum does not correspond to PL 1.0 but to some pi < 1.0.This implies that in the case the velocities of the leader and the agents are different the maximum efficiency of the collective movement is achieved if there is some noise in the leader/agents communication.In other words excess of infor- mation from the leader makes the coordination decreases as far as the realization of the collective movement is concerned.
Figure 6 presents the variation of the amplitude intensity of the main peak against PL for vL =0.8 and R=50 (circles), R=75 (triangles).The maximum corresponds approximately to p 0.1.In Fig. 7 the same curve is presented for a different case where the left, right and ahead perception sectors are not all equal: vL=0.8,Ra--0.3,Rl=Rr 100.Here the amplitude is maximum around pi 0.3.
The corresponding value of p< 1.0 for the maximum amplitude may be interpreted as a counterbalance with respect to the relative velocity if we interpret the parameter vI as an advanced response of the agents to the leader's signal.Ifp is large the group of agents following the leader moves in a less coherent way with respect to the collective pattern.As a consequence the level of noise in the correlation function increases and the amplitude corresponding to the collective movement de- creases.This effect is due exclusively to the inter- action between the leader and the group of agents together with the communication between agents.
3.4 Dependence with Respect to R and RL The dependence of the collective movement with respect to R is simple.For fixed values of v > 0.5, pL and R we observe an increase in the amplitude intensity of the corresponding collective movement with the value of R.This means that the farest the signal of the leader is perceived in space the largest the efficiency of the corresponding collective movement.
In order to understand the role of R and RI for the efficiency of the collective movement we con- sidered two cases.In Fig. 8 it is shown the variation of the amplitude intensity with p for vL 0.8, R 200, R 50, and vI 0.8, R 50, R 200.
For small probabilities the collective pattern is better performed if the agents are capable to communicate in large distances.If the agents can communicate only at shorter distances then to improve the efficiency of the collective pattern the probability p has to increase.FIGURE 8 Amplitude of the main peak as a function of PL for vL=0.8 for the cases R1=200, R=50 (triangles) and R 50, R 200 (circles).

PATTERN RECOGNITION PROBLEM
From the construction of the model it is clear that the agents follow the leader without any knowledge about the shape of the leader's path.
The set of ordinary agents may be used to identify the shape of the leader's path by tracking its path in time.For that it is defined a "track finder" in the following way: At each time step it is identified the subgroup of ordinary agents which has perceived the leader in the perception region and is updated directly by the leader's rule.Next the position of the baricenter of this subgroup is calculated.The baricenter follows the leader approximately on the same path depending on the free parameters vi, pi, RL, R. Keeping all the parameters but pi fixed the system will move according to the collective move- ment associated with the value of v.
Clearly the position of the baricenter of the sub- group will depend on the coordination with which the collective movement is performed through the parameter PL.The existence of a dynamically im- plemented collective movement implies that the baricenter should also perform a movement in time with some regularity reflected by the corresponding velocity correlation function.Therefore the question is how efficiently the baricenter tracks the path of the leader.finder with respect to the radius of the leader's path.The minimum around PL 0.3 shows that the track finder opti- mally follows the leader's path in the situation where the coordination of the collective movement is maximal.
We have calculated the mean distance in time of the baricenter to the center of the leader's circular path as a function of p for the fixed configura- tion of the remaining parameters Ra--30, RI= Rr 100 and v 0.8.The percentage deviation of the position of the track finder (baricenter) with respect to the radius of the leader's path is plotted in Fig. 9.
The minimum deviation is around p 0.3, the value of pL which in this case corresponds to the maximal coordination of the collective movement.Therefore the pattern of the leader's path is opti- mally recognized by the track finder in the case of maximal coordination between the ordinary agents.
This fact reinforces the interpretation that the coordination ofthe realization ofthecollectivemovement is a result of the flow of information through the group of autonomous agents.As the agents are identical the noise in the communication, simulated by p, is determinant for the coordination of the collective movement for the whole group of agents.
The example presented here of the construction of a track finder is just one example of a quantity with a practical significance which depends directly on the coordination with which the collective move- ment is performed.An important case is to consider the group of ordinary agents composed by different individuals, for instance with respect to the size of the perception sectors.which intercommunication between parts plays a crucial role. 5

CONCLUSION
We have studied a model of communicating agents which are probabilistically coupled with a leader agent performing a stable and periodic movement.
As a result of the coupling different collective behaviors emerge from the elementary individual interactions between the agents.
Depending upon the velocity difference between the agents and the leader agent the system moves in a coordinated way adapted to the rescaled periodic information generated by the leader agent.
We explore the fact that the leader may be seen as a source of information which is sent to the group of autonomous agents.The group can trans- form this information through communication be- tween agents.The probability coupling that exists between the group of agents and the leader agent simulates a noise in this information transfer.
The probability coupling also determines the efficiency or coordination of the emerging collective behavior.Nontrivial values of the probability are needed for the optimal performance of the collec- tive movement.The range in which the agents may communicate is another factor that may change this coordination.We observe that the efficiency of the collective movement increases with the increasing range of the communication of the agents.We also observe that in certain cases to increase the range of communication between the leader and the agents works as an additional source of noise decreasing the coordination with which the collective move- ment is performed in time.
The system discussed here is a simple model for the dynamics of intercommunicating systems submitted to a noisy but periodic and stable per- turbation.We conjecture that the observed results like the importance and interplay between noise (PL) and anticipated response (vL) represent more than a mere specificity of the model and should be related to similar phenomena in real systems for

FIGURE 3
FIGURE 3 The velocity correlation function for the case vL =0.8.

FIGURE 4
FIGURE4 Power spectrum of the velocity correlation func- tion for vi 0.8.The main peak corresponds to the frequency of the collective movement of the agents.

FIGURE 6 FIGURE 7
FIGURE 6 Amplitude of the main peak of the power spec- trum as a function of PI.The maximum around PI 0.1 indi- cates the situation of optimal coordination of the collective movement.

FIGURE 9
FIGURE 9 Percentage deviation of the position of the track