Modeling and Analysis of a Simple Manufacturing-Oriented Multi-Agent System

Due to the autonomy of individual agents and the use of the concept of distributed planning, multi-agent systems (MAS) represent a promising approach to achieve fault-tolerant self-organizing manufacturing systems. In this article, a basic component of a manufacturing-oriented MAS is presented. The negotiation strategies are formulated in such a way that they, on the one hand, guarantee considerable flexibility of the basic component itself, and, on the other hand, enable the construction of more complex systems built up from several components. On the basis of this single component, it is shown that the dynamics of such systems without appropriate control mechanisms can be chaotic. Such behaviour is, however, unwanted in practice and must therefore be stabilized or avoided. In order to develop appropriate tools for this task, the dynamic behaviour of the system is investigated using concepts and methods of synergetics and the theory of nonlinear dynamical systems.


MOTIVATION AND PROBLEM DEFINITION
In the current economic situation, in which custom- ized series manufacturing is more and more important for successful competition, flexible manufacturing systems are targetted, which can be adapted to new variants of the product rapidly and without large expenditure (see, e.g., Warnecke,  1992).The use of multi-agent systems (MAS) is recommended for this purpose because the auton- omy of individual system components (intelligent manufacturing robots) and the use of the concept of distributed planning guarantee large flexibility and fault-tolerance of such systems.However, they can also cause complex dynamic behaviour (periodical, quasi-periodical, chaotical) of the system (Levi, 1989).
Thus, in order to enable the application of MAS for manufacturing, the dynamics of these 36 P. LEVI et al.
systems and the eligible control strategies must be thoroughly examined.The current focus of our in- vestigations is, therefore, firstly, modelling of such systems and, secondly, development of suitable analysis methods, whereby concepts of synergetics (Haken, 1983a,b), and the theory of nonlinear dynamical systems (Wiggins, 1990; Guckenheimer  and Holmes, 1993; Arnol'd et al., 1994) are used.

SYNERGETICS
Synergetics is an interdisciplinary science, which deals with the investigation of a special class of dynamical systems.These systems are open systems.which consist of many interacting subsystems.The systems can be affected from outside by so-called control parameters.Due to self-organizing pro- cesses on the microscopic level, these systems show ordered structures (temporal, spatial or functional) on the macroscopic level which we call patterns.The ordered structures are often qualitatively stable for large ranges of the control parameters.Only at critical values of the control parameters, the so- called instabilities, the behaviour of these systems changes qualitatively and new macroscopic ordered structures emerge.It is shown in (Haken, 1983a,b), that in the vicinity of an instability, the macroscopic ordered structures, even for very complex systems, can be described by a few degrees of freedom, the so-called order parameters.If one investigates systems, for example from the field of physics, then the underlying microscopic equations of motion and the associated self-organizing mechanisms are given by laws of nature.Our intention in this context is to apply the methods and concepts of synergetics to our manufacturing-oriented MAS.Thus, in terms of synergetics, the negotiation strategies of a MAS represent the microscopic equations of motion and the resulting interaction patterns, the macroscopic ordered structures.The negotiation strategies gen- erally contain certain parameters, which we can identify as the control parameters of the system.As expected, our investigations show that the interac- tion patterns of MAS also change qualitatively at certain critical values of the control parameters, as is the case in, for example, physical or chemical systems.When dealing with MAS we can design the negotiation strategies and therefore have, in terms of synergetics, great influence on the microscopic equations of motion and hence, ultimately, on the emerging ordered structures on the macroscopic level.From this point of view, our task then is to design the negotiation strategies in such a way that desired patterns of inter-agent interaction emerge.
To illustrate the principle, we shall consider a fault- tolerant manufacturing-oriented MAS.We identify the production of a certain product with a macro- scopic ordered structure of the manufacturingoriented MAS.The fault-tolerance requires that this production can successfully continue even in case of a breakdown of one or several manufacturing robots.The negotiation strategies have to be designed in such a way that the agents involved not only register the breakdown of one or several robotic units but also initiate, if necessary, a new planning phase, in order to compensate for the failure as fully as possible.Obviously, in general, the total production will decrease thereby.Contrary to fixed manufacturing systems, a manufacturing- oriented MAS with such negotiation strategies implemented is capable of a self-organized adaption to changing situations.The disadvantage of these flexible negotiation strategies is the fact that they could cause a chaotic dynamic behaviour of the MAS.Of course, such behaviour must be avoided.Thus, the negotiation strategies must be extended by appropriate control mechanisms.The theory of nonlinear dynamical systems can con- tribute to these promising concepts and methods which, however, must be adapted and extended for the application in MAS.

NEGOTIATIONS
Until now there exists no general concept how one can design a manufacturing-oriented MAS that fulfils all the requirements mentioned above.The underlying idea of the work presented in this article is the synthesis of the multi-agent concept (Fischer, 1994; Levi and Hahndel, 1995), on the one hand, and the concepts of synergetics, on the other hand (Haken, 1983a,b).While the multi-agent con- cept enables the flexibility and the distributed planning, the synergetics provide methods which allow the systematic investigation of self-organization mechanisms in these systems.Our goal is it then to implement such self-organization mechanisms into the negotiation strategies of a manufacturing- oriented MAS, thus enabling a stable, flexible and efficient mode of operation.Since in a MAS there exists no central planning unit, all planning deci- sions (i.e. the decisions on which robotic unit has to execute which manufacturing operations end in which order) have to be done by negotiations among the agents.Thus, a negotiation strategy determines when and for which functions indi- vidual agents communicate with each other.The decisions thus take place according to a given opti- mization criterion, e.g. the minimization of cost or execution time.In this article a task is regarded as a unique entity, which cannot be divided between several agents.For this reason, modelling of such type of systems requires the use of the so-called O-functions (indicator functions, decision func- tions, step functions; see Section 4.1.2) which are differentiable only in the sense of distribution theory.By applying these functions, uniqueness of the decisions can be guaranteed.In process of the negotiations the agents adopt different roles.The agent which sends inquiries to other agents in order to decide to which of these agents a current task should be sent, is called a manager because it ini- tiates a negotiation.The agents to which it sends these inquiries, will be referred to as robotic units.Additionally we distinguish between two types of negotiation strategies: Vertical negotiations: The manager negotiates with each robotic unit which is capable of per- forming the current task and then comes to the planning decision itself.
Horizontal negotiations: A task is sent first to a specific group of robotic units capable of performing that task.The robotic units then come to the planning decision by negotiating about the task among themselves.
In order to be appli.able in practice, a negotia- tion strategy must contain some aspects from both types.Since it is to a large extent unknown how one can examine complex combinations of the negotia- tion strategies described above, we begin our anal- ysis with very simple systems and proceed further in steps: (1) Investigation of an extremely small system consisting of three agents (one manager and two robotic units), which can serve as a basic component for more complex manufacturing- oriented MAS.Crucial parameters for the dynamic behaviour of the system have to be identified.Thereby the model is designed in such a way that it can serve as a basic com- ponent for more complex MAS, in which both vertical and horizontal negotiations are implemented.
(2) Investigation of a larger MAS with vertical negotiation strategy and examination or exten- sion of the methods developed in the first stage.
(3) Further extension to a model with still more complex characteristics and abilities (horizontal negotiations, stochastic events, ...), in order to achieve negotiation strategies which are more relevant for practical purposes.
In this article we present our model and the first results of our investigations.Because this model serves as a basic component for the construction of more complex manufacturing-oriented MAS it is worth mentioning here that further investigations of this simple model have to be performed in order to achieve detailed knowledge of the effects occur- ing in such systems.

BASIC MODEL: M1R2-SYSTEM
WITHOUT FEEDBACK First, we define the system which we intend to model and which will be analyzed in the following.

DEFINITION
A M1R2-system is a negotiation- based manufacturing-oriented MAS which consists of three agents (one manager and two robotic units).
The robotic units can process similar repetitive tasks which they get from the manager at certain times.
By negotiating with the robotic units, the manager gets information about their states and decides, depending on this information, which of the robotic units has to perform the task (Fig. 1).

State Variables
Our intention is to make quantitative analysis of the dynamic behaviour of manufacturing-oriented MAS possible.Therefore, we have to model the sys- tem in a mathematical way, that is, to formulate the equations of motion of the states of the robotic units.In order to do so, we must first clarify what we denote as a state of a robotic unit.One way, which we pursue in this article, is it to regard the contents of the input buffers of the robotic units as state variables.
DEFINITION 2 The function 0 of the input buffer content of the robotic unit R (i 1,2) indicates the amount of work which has to be executed by the robotic unit.We assume in our model that a task is represented by a positive rational number C, which indicates its size.Then, clearly, the value pi at each time is the difference between the total number of tasks assigned to the robotic unit R (i= 1,2) and the number of tasks it has finished up to this time.
The function Pi is defined on the finite interval [0, Dimax], where Di max denotes the size of the buffers of the robotic units.

Processing of Assigned Tasks
To define the equations of motion we have, in prin- ciple, two possibilities, namely, we can use either a time-discrete or a time-continuous formulation.
Because we have defined simple tasks which cannot be divided into subtasks, it is natural to formulate the equations of motion of the M1R2-System in terms of time-discrete difference equations.If the robotic units operate with constant productivities ai, which normally should be the case, then the buffer contents pi(n) at the discrete time n are reduced by the values of O within one time step.
Of course it has to be recognized that the buffer contents remain positive or at least zero.If we use  n --[-1) (/gi(n) ozi)[ oi (n > ozi] (2) as equation of motion of the buffer content pi(n).According to this evolution equation, a robotic unit processes the content of its buffer, i.e. the assigned tasks, until its buffer is empty.

Assignment of Tasks
The buffer content pi(n) is increased by C if the robotic unit Ri receives a task of a size C at time n.With #(n) we denote a function which takes the value if the manager sends a task to the robotic unit at the time n, and otherwise takes a zero value.Another function, which takes the value if a task is assigned to the robotic unit Ri and zero otherwise, is called Hi[ ].Substituting these notations into Eq.(2), we obtain: We call the function #(n) from the last paragraph the manager function.The simplest form of this function, which we consider first, describes a situ- ation when the manager sends the tasks to the robotic units with a constant frequency a, i.e. the function #(n) takes the value with a certain period T 27r/cv.Hence we can define it as In order to define the functions Xi[], we use the following time optimization criterion: a task is assigned to the robotic unit which can complete it most rapidly.Therefore, we have to compare the processing times of the robotic units.If a task of size C is assigned to a robotic unit at time n, it takes the time (pi(n)+ C)/ci, to process it.*In addition, it must be guaranteed that the boundary condition pi(n if-1) < p/max is satisfied when the robotic unit Ri gets the task.Thus, a robotic unit receives a task if: (1) There occurs no overflow of its buffer (cond. 1) when receiving the task (2) It can process the task faster than the other robotic unit (cond. 2),or the buffer of the other robotic unit will overflow when receiving the task (cond.3).
It is easy to prove that the following functions fulfil these requirements" Xi[]-X[Pi(/7) at-C <_/di-max cond.1 with i,j-1,2 and i:j.
4.2 Dynamic Behaviour of the M1R2-Model without Feedback Equations (3)-( 5) define the complete M1R2- model without feedback.It turns out (see Levi  et al., 1998) that this simple model shows an inter- esting dynamic behaviour (see Fig. 2).Therefore one has to recognize that the total content of the buffers =1 pi(n) is increased within one time step by C/T on average and the robotic units process Y]=l ci task units in the same time.One can show that the dynamic behaviour of the system depends qualitatively on the values of C and T.There are three different regions in the parameter space, namely where we have assumed c1 > Ct2 without loss of generality.
Under the prerequisite that the buffers of the robotic units are large enough, the first two cases correspond to a periodic behaviour of the system.In the first case, the period is exactly T (see Fig. 2(a)), and the system is permanently under-loaded so that only the faster robotic unit works.In the second case, the period is a multiple of T (see Fig. 2(b)-(d)), and both robotic units work.In the third case, the system shows a quasi-periodic behaviour (see Fig. 2(e)-(g)).In this case both robotic units work, but the systems capacity is exceeded, that means the buffer contents pi(n) reach their maximal values, which leads to a situation when some tasks are rejected by both robotic units.Thus, c0-- 27r/C,_ ci is the optimal task frequency in the M1R2-model without feedback (3).

EXTENDED MODEL: MIR2-SYSTEM WITH FEEDBACK
The model presented so far possesses a substan- tial disadvantage.The manager function #(n) represents a central planning authority, which decides which of the robotic units gets the next task according to the current situation, but is not able to modify the task frequency.However, such a modi- fication is necessary in order to ensure long-term optimization of the manufacturing process.If the task frequency is set to the optimal value COo at the beginning, it must be modified externally if the size C of the tasks or the productivities of the robotic units ci change, for example due to a breakdown of a robotic unit.Therefore, it is necessary to extend the model in such a way that according to the cur- rent states of the robotic units they are able to inform the manager that the task frequency should be modified.The difficulty here is that such information can be contradictory and it is not clear until now how to find an optimal strategy which satisfies these different demands to some extent simultaneously.

Definition of the Extended Model
General Representation of a Feedback Function In mathematical terms, we want to implement a feedback loop, so that the task frequency is con- trolled by the system itself.The definition range of the frequency CO is therefore the interval (0, COma].
The general representation of a class of feedback strategies is the following: CO(n + 1) --f(CO0, co(n)) + g(pl(n),pz(n)) (9)   with a control function f(CO0, CO(n)) and a feedback function g(pl(n),pz(n)).This function takes into account a priori knowledge about the control parameters of the system, i.e., the productivities O of the robotic units and the size C of the tasks.In this paper, the specific version X )(, 0 < CO(fl)+ /CO(pi(fl)) COmax i--1 is used.The one-dimensional feedback function ACO(pi(n)) corresponds to the case of the robotic units R1, R2 not communicating with each other but only with the manager.The value of this function is interpreted as the request from one individual robotic unit, how the frequency CO(n) of the tasks should be modified according to its buffer content.The decision function guarantees that the manager always considers these requests if the modification does not cause a violation of tile definition range (0, conax] of the frequency.

One-Dimensional Feedback Function
When designing the function Aco(pi(n)) one has to consider the following two situations: (1) This means that the buffer of the robotic unit is empty at the next time step if no new task arrives within this time.The robotic unit concerned will thus request an increase of the task frequency (2) pi(gl) -C > p?aX (12)   This means that the buffer of the robotic unit is full and the robotic unit cannot receive a new task as long as this condition holds.In this case it is clear that the robotic unit concerned will request a decrease of the task frequency, i.e., Aco(pi(n)) must be negative.To summarize, we obtain: In the investigations carried out in this paper, the feedback function represented in Fig. 3 is used.Of course there exist several possibilities for the design of such feedback functions and the one which we have chosen here is more or less arbitrary.
If the condition (11) holds, the current value of the task frequency has to be increased by the value of Av+, whereas if the condition ( 12) is fulfilled it has to be decreased by the value A0v-.In the region between c; and C-p/ma the function is defined in such a way, that a fine tuning takes place and at the ^opt optimal value t,; d/kco(fli(n)) pi(tl)=pTpt =0 holds.

The Resulting Set of Equations
The M 1R2-model with feedback is described by the following set of time discrete difference equations: with 1,2, k E N and the function Xi[] as defined in Eq. ( 5).It is obvious, that the Eqs.( 16) and ( 17) represent an open nonlinear time-discrete dynamical system.In Fig. 4, a typical trajectory of the system is shown.
As one can see the system shows, after a transient region, chaotic dynamics.In fact, we are not able to localize any areas in the parameter space where the system would indicate either a stationary, a periodic or a quasi-periodic dynamics.Therefore, we expect that without a suitable modification or extension, this system can only show a chaotic dynamic behav- iour, which is caused by the implemented flexible 7,//,,/,' " . .s : ./",,, .' ,/(,. -" ":"i.. ,; . . . . .FIGURE 5 Phase portraits of the order frequency co(n) at different productivities of the robotic units.Parameter values: C 1, pl(O)--O, /92(0)--0, p{nax 15, pnax 15, ppt= 7.5,pPt--7.5,co(O) 10-4, comax 2, /kco + --0.0004,Aco-0.0004,(a): a, 0.002, O 0.05, (b): c1 0.02, c2 0.05.negotiation strategy.One can also see from Fig. 4 that there is a remarkable correlation between the trajectories pi(n) and co(n), which is to be expected in accordance with the construction of the system.In addition, it can be seen that the dynamics of the task frequency co(n) determine the buffer contents pi(n).From the point of view of synergetics, the behav- iour of co(n) is typical for an order parameter in the vicinity of an instability.We therefore assume that the flexibility of our MAS causes the system to be always in the vicinity of an instability.
When looking at the phase portrait Aco(co(n)) (Fig.  4(d)) one can observe that the trajectories of the system form a confined, complex, object in the phase space [0, comax] X [-2Aco-, 2A+]; which means that they could belong to a strange attractor.The centre of this object corresponds to the optimal task frequency coo 27r/C = ci of the M 1R2- model without feedback.
In Fig. 5(a) we have chosen the parameters ofciin such a way that the optimal task frequency is located in the lefthand side of the phase space.Conversely,   Fig. 5(b) corresponds to the optimal task frequency being located on the righthand side of the phase space, but outside the range which is given by the value of comax.As one can see, this results in a cor- responding shift of the attractor-like object. 6

SUMMARY
In this work, two models of a simple manufactur- ing-oriented MAS were introduced, namely the M1R2-model without feedback (3)-( 5) and with feedback ( 16) and ( 17).
Depending on the control parameters, i.e., the productivities OZ of the two robotic units, the size C of the tasks, and the task frequency co, the variant without feedback is either under-loaded, works optimally or is over-loaded (see Section 4.2 for details).The M 1R2-model without feedback shows a periodic dynamic behaviour, which corresponds to a constant productivity and is therefore suitable for manufacturing-oriented MAS, but it is not flexi- ble enough to adapt by self-organization to changes of the size C of the tasks.
Hence, to improve flexibility, we introduced a feedback mechanism into the model.Although the negotiation strategy of the model with feedback is still very simple, it has practical relevance, because a time optimization criterion is considered as well as real constraints such as, for example, the finite buffer sizes.

OUTLOOK
One important result of our investigations is that the mathematical modelling of close-to-reality manu- facturing-oriented MAS leads to nonlinear evolu- tion equations with complex couplings of the state variables in which decision functions are involved (see Section 4.1.2).These functions cause very complex problems for analytical treatment of such systems, therefore further work has to be carried out in order to discover the basic features and characteristics of the interactions among the auto- nomous agents, because this knowledge is a neces- sary prior condition for designing efficient negotiation strategies in flexible manufacturingoriented MAS.As already mentioned, one can observe a chaotic dynamic behaviour of our simple M1R2-model with feedback within large areas in the parameter space.This has, from our point of view, mainly two consequences: (1) It is to be expected that larger and more com- plex manufacturing-oriented MAS with imple- mented self-organization principles also show chaotic dynamics.
(2) To enable the application of MAS in flexible manufacturing, this chaotic behaviour has to be controlled or even avoided, in order to achieve a stable stationary oscillatory dynamic behaviour which corresponds to a regular and in some sense optimal mode of operation.Hence, suit- able and efficient control mechanisms have to be developed, analysed and implemented in the negotiation strategies.
In future work we will therefore, at a first step, try to stabilize or control the chaotic dynamics; to this end we intend to use the well-known methods of nonlinear dynamical systems, like those proposed in (Ott et al., 1990) or (Pyragas, 1992; 1993).These methods have to be adapted and investigated thoroughly before implementing them into the negotiation strategies.Concerning the numerical analysis it is our intention to use concepts which are based on the theory of nonlinear dynamical sys- tems, such as calculation of Lyapunov exponents (Oseledec, 1968; Wolf et al., 1985) or complexity analysis using symbolic dynamics (Hao, 1991).As regards analytical investigations, we plan to make use of the well-known concepts of synergetics and their extensions to delayed feedback systems pre- sented in (Wischert et al., 1994; Schanz, 1997).
At a second step we shall focus on more complex planning situations, which lead to combinatorial optimization problems.To solve these problems, we can make use of well-known techniques such as simulated annealing (van Laarhoven and Aarts,   1987) or the stochastic net method (Fort, 1988), but also of more recently developed ones, based on the pattern-formation is pattern-recognition analogy of synergetics (Starke, 1997; Starke and Schanz, 1998).
Finally, we shall have to enlarge the system and to introduce stochastic variables in order to simu- late imprecise planning situations or malfunctions in the manufacturing-oriented MAS, such as changes in the production chains or sudden break- downs of robotic units.At this stage, we shall also have to apply stochastic analysis in order to calcu- late corresponding distribution functions.
All these topics contribute to our main goal which consists in developing generalized rules and concepts of design of negotiation strategies in manufacturing-oriented MAS to enable production to be self-organized and hence as flexible as possible.

FIGURE
FIGURESchematic representation of the M1R2-model.

FIGURE 2
FIGURE 2 Behaviour of the M1R2-model without feedback.

FIGURE 3
FIGURE 3 Investigated feedback function.

5. 2
Dynamic Behaviour of the M1R2-Model with Feedback