Stochastic Algorithms for Buffer Allocation in Reliable Production Lines

The allocation of buffers between workstations is a major optimization problem faced by manufacturing systems designers. It entails the determination of optimal buffer allocation plans in production lines with the objective of maximizing their throughput. We present and compare two stochastic approaches for solving the buffer allocation problem in large reliable production lines. The allocation plan is calculated subject to a given amount of total buffer slots using simulated annealing and genetic algorithms. The throughput is calculated utilizing a decomposition method.


Introduction
The allocation of buffers between workstations is a major optimization problem faced by manufacturing systems designers.It has to do with devising an allocation plan for distributing a certain amount of buffer space among the intermediate buffers of a production line.It is a very complex task that must account for the random fluctuations in mean production rates of the individual workstations of the lines.To solve this problem there is a need for two different tools.The first is a tool that calculates the performance measure of the line which has to be optimized (e.g., the throughput or the mean work-in-process).This may be an evaluative method such as simulation, a decomposition method [1,2], or a traditional Markovian state model in conjunction with an exact numerical algorithm [3].The second tool is a search (generative) method that tries to determine an optimal or near optimal value for the decision variables, which in our case are the buffer capacities of the intermediate buffer locations in the line.Examples of such methods are the classical search methods such as the well-known Hooke-Jeeves method, various heuristic methods, knowledge based methods, genetic algorithms, and simulated annealing.
Evaluative and generative (optimization) models can be combined in a 'closed loop' configuration by using feedback from an evaluative model to modify the decision taken by the generative model.In such a configuration the evaluative model is used to obtain the value of the objective function for a set of inputs.The value of the objective function is then communicated to the generative model which uses it as an objective criterion in its search for an optimal solution.In the rest of this paper we will use the formalism S(G E ) to describe a closed loop system using the generative method G and the evaluative method E .The generative models that will be used in this paper are:

CE complete enumeration,
Mathematical Problems in Engineering, 5:441-458, 2000.y This is a machine-readable rendering of a working paper draft that led to a publication.The publication should always be cited in preference to this draft using the reference in the previous footnote.This material is presented to ensure timely dissemination of scholarly and technical work.Copyright and all rights therein are retained by authors or by other copyright holders.All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright.In most cases, these works may not be reposted without the explicit permission of the copyright holder.

RE reduced enumeration,
GA genetic algorithms, and SA simulated annealing.Furthermore, two evaluative models will be used: Exact the exact numerical algorithm described in the work by Heavey at al. [3], and Deco the decomposition algorithm numbered as A3 in the work by Dallery and Frein [2].
An overview of the existing research in the area of evaluative and generative models of manufacturing systems can be found in two review papers [4,5] and a number of books [6,7,8,9,10,11].
Several researchers have studied the problem of optimizing buffer allocation to maximize the efficiency of a reliable production line [12,13,14].Their results are based on comprehensive studies to characterize the optimal buffer allocation pattern.Authors have provided extensive numerical results for balanced lines with up to 6 stations and limited results for lines with up to 9 stations.However, few methods can handle this problem for large production lines, in a computationally efficient way.In this paper we compare two stochastic approaches suitable for large production lines, one based on genetic algorithms, and one based on simulated annealing.Details on how these methods can be applied to the problem can be found in the work by Bulgak et al. [15] which describes the application of genetic algorithms for the buffer allocation in asynchronous assembly systems and in our work [16] which describes a corresponding approach using simulated annealing.The implementation of both approaches in this paper works in close cooperation with a decomposition method [2].
Simulated annealing is an adaptation of the simulation of physical thermodynamic annealing principles [17] to the combinatorial optimization problems [18,19].Similar to genetic algorithms and tabu search techniques [20] it follows the "local improvement" paradigm for harnessing the exponential complexity of the solution space.The algorithm is based on randomization techniques.An overview of algorithms based on such techniques can be found in the survey by Gupta et al. [21].A complete presentation of the method and its applications is described by Van Laarhoven and Aarts [22] while a number of works present accessible algorithms for its implementation [23,24].As a tool for operational research simulated annealing is presented by Eglese [25], while Koulamas et al. [26] provide a complete survey of simulated annealing applications to operations research problems.
Genetic algorithms [27,28,29] are global optimization techniques that avoid many of the shortcomings exhibited by local search techniques on difficult search spaces, such as the buffer allocation problem.Goldberg [30] describes a number of diverse genetic algorithm applications, while Karr [31] presents their use for modelling, design, and process control.Finally, Tompkins and Azadivar [32] use genetic algorithms for optimizing simulated systems.
This paper is organized as follows: Section 2 states the problem and the assumptions of the model and Section 3 describes the evaluation methodology and associated implementation decisions.In Section 4, we compare the numerical results obtained from the algorithms.Finally, Section 5 concludes the paper and suggests some future research directions.

The Buffer Allocation Problem
In asynchronous production lines, each part enters the system from the first station, passes in order from all stations and the intermediate buffer locations, and exits the line from the last station.The flow of the parts works as follows: in case a station has completed its processing and the next buffer has space available, the processed part is passed on.Then, the station starts processing a new part that is taken from its input buffer.In case the buffer has no parts, the station remains empty until a new part is placed in the buffer.This type of line is subject to manufacturing blocking (or blocking after service) and starving.

Assumptions of the Model
The model operates under the assumption that the first station is never starved and the last station is never blocked.The processing (service) times at each station are assumed to be independent random variables Station i

Buffer i
Figure 1: A K -station production line with K ; 1 intermediate buffers following the exponential distribution, with mean service rates, i , i = 1 2 : : : K .In our model, the stations of the line are assumed to be perfectly reliable, that is, breakdowns are not allowed.
The exponentiality of the processing times as well as the absolute reliability of the line's workstations are rather unrealistic assumptions.However, the service completion times can be exponential or can be approximated by an exponential distribution.The variability in completion times may be attributed to failures and repairs which implicitly exist in the problem at hand.Following this view, the proposed model may be applied to any unreliable production line under the exponentiality assumptions for the service completion times.
Figure 1 depicts a K -station line that has K ;1 intermediate locations for buffers, labeled B 2 B 3 : : : B K .
The basic performance measures in the analysis of production lines are the throughput (or mean production rate) and the average work-in-process (WIP) or equivalently the average production (sojourn) time.
The object of the present work is the buffering of asynchronous, reliable production lines with the assumptions given above.The objective is the maximization of the line's throughput, subject to a given total buffer space.

The Buffer Allocation Problem
In mathematical terms, the buffer allocation problem can be stated as follows: subject to: where: N is a fixed nonnegative integer, denoting the total buffer space available in the production line, B = (B 2 B 3 : : : B K ) is the 'buffer vector', i.e., a vector with elements the buffer capacities of the K ; 1 buffers, and O K , denotes the throughput of the K -station line.This is a function of the mean service rates of the K stations, i (i = 1 2 : : : K ), of the coefficients of variation, C V i , of the service times and the buffer capacities, B i .
The number of feasible allocations of N buffer slots among the K ; 1 intermediate buffer locations increases dramatically with N and K and is given by the formula: For this reason exhaustive search techniques are not practical for determining optimal configurations of production lines with a large number of stations or buffers.

Evaluation Methodology
We have evaluated different approaches for solving the optimal buffer allocation problem for large production lines, by performing the following steps: S1 We utilized the decomposition method [2] as an evaluative tool to determine the throughput of the lines.
The algorithm computes approximately the throughput for any K -station line with finite intermediate buffers and exponentially distributed processing times.

S2
To find the buffer allocation that maximizes the throughput of the line, we utilized two stochastic methods, simulated annealing and genetic algorithms, specifically adapted for solving this problem.
In order to evaluate the applicability of the stochastic methods to the buffer allocation problem using comparable architectures we designed and implemented a system to calculate the optimum buffer configuration for a given reliable production line using a variety of algorithms [33].The system takes as input: the number of stations in the production line, K , the available buffer space, N , and the station mean service rates, i , i = 1 2 : : : K .Based on the above input, the system calculates the buffer allocations B = (B 2 B 3 : : : B K ) for the maximal line throughput.Furthermore, the system is instrumented to provide as part of the solution the throughput of the suggested configuration, as well as the number of different configurations that were tried.The line throughput is used to evaluate the quality of the suggested configuration when compared with the throughput calculated by other methods.The number of different configurations tried, is used as an objective performance criterion, because the configuration evaluation step is the dominant execution time factor and the basic building block of all optimization methods.In addition, a special system configuration allows the creation of a file containing step-by-step snapshots of the algorithm progress.After obtaining the test results we wrote a number of scripts in the Perl programming language [34] that utilized the snapshot file to visualize and animate the dynamic behavior of the algorithms.
We ran a number of tests on both balanced and unbalanced lines and compared the stochastic method results against each other and against the results obtained by other methods.For short lines and limited buffer space a complete enumeration of all configurations provided an accurate measurement base to verify the stochastic algorithm results.For larger configurations we used a reduced enumeration in order to provide the comparative measure.

The Reduced Enumeration Method
Reduced enumeration is based on the experimental observation that the absolute difference of the respective elements of the optimal buffer allocation (OBA) vectors with N and N + 1 buffer slots is less than or equal to 1: In this way, we have been able to derive the OBA by induction for any number N of buffer slots that are to be allocated among the K ; 1 buffer locations of the line.The reduction works as follows: when N and K are given one needs to determine all the OBA vectors for N = 1 2 : : : N and then for N = N + 1 by searching only the values of B N i ; 1, B N i and B N i + 1 .Furthermore, this reduction starts after a number of total buffer slots N .To quantify the reduction, by applying the improved enumeration it has been experimentally observed that the number of iterations were reduced by at least 60% for short lines.This reduction accounts for well over 90% for large production lines (with more than 12 stations).Recall that the number of feasible allocations of N buffer slots among the K ; 1 intermediate buffer locations increases dramatically with N and K and is given by Formula (3).

Simulated Annealing
Simulated annealing is an optimization method suitable for combinatorial minimization problems.Such problems exhibit a discrete, factorially large, configuration space.In common with all paradigms based on "local improvements" the simulated annealing method starts with a non-optimal initial configuration (which may be chosen at random) and works on improving it by selecting a new configuration using a suitable mechanism (at random in the simulated annealing case) and calculating the corresponding cost differential ( O K ).If the cost is reduced, then the new configuration is accepted and the process repeats until a termination criterion is satisfied.Unfortunately, such methods can become "trapped" in a local optimum that is far from the global optimum.Simulated annealing avoids this problem by allowing "uphill" moves based on a model of the annealing process in the physical world.
Our implementation of the simulated annealing algorithm for distributing N buffer space in a K -station line [16] follows the following steps: 1. Set initial line configuration.]Set B i b N = Kc, set B K=2 B K=2 + N ; P K i=2 bN=Kc.

Genetic Algorithms
Genetic algorithms are also global optimization techniques that avoid many of the shortcomings exhibited by local search techniques on difficult search spaces.They rely on modeling the problem as a population of organisms.Every organism represents a possible valid solution to the problem.Organisms are composed of alleles representing parts of a given solution.Standard genetic recombination operators are used to create new organisms out of existing ones by combining alleles of the existing organisms.In addition, mutations can randomly change the composition of existing organisms.Typically, the algorithm evaluates all the organisms of the population and creates new organisms by combining existing ones based on their fitness.This procedure is repeated until the variance of the population reaches a predefined minimum value.
An important characteristic of our implementation of the genetic algorithm concerns the representation of the solution.A good representation should ensure that the application of standard crossover recombination operators (where a new organism is composed from parts of two existing ones) would result in a valid new representation.Representing the line configuration as a vector B of buffers allocated across the line is not such a representation since given two buffer configurations (B 1 B 2 ) and recombining them as a new buffer B 0 at point c so that B 0 0:::c B 1 0:::c and B 0 c+1:::K B 2 c+1:::K will not guarantee that P K i=1 B 0 i = N i.e. that the resulting line configuration will be composed of N buffers.For this reason we devised an alternative, position-based, representation using a vector P of length equal to the number of buffers N .Every element of P can take values 0 : : : K representing the position of the given buffer slot within the production line.The two representations are equivalent; the vector P can be mapped to B as follows: The position-based representation will generate valid buffer configurations using standard genetic crossover and mutation operators.Using this representation, the genetic algorithm we implemented for distributing N buffer space in a K -station line is described in the following steps: 1. Initialize a population of size S .]Set P 0:::S 0:::N b rand 0 : : : K ; 1)c.
6. Keep fittest organism for elitist selection strategy.]Select f so that T f T 0:::S , set P 0 brand 0:::S)c P f .7. Make new population the current population.]Set P P 0 .
8. Loop based on the population's variance.]If P P i=0 jT f ; T i j > maximum variance go to step 2; otherwise the algorithm terminates with the optimal line setup in P f .
The implementation of genetic algorithms can be tuned using a number of different parameters.In our implementation we used the parameters that Grefenstette [35] derived using meta-search techniques namely: a population size S of 50, a crossover rate of 0:6, a mutation rate of 0:0001, a generation gap of 1 (the entire population is replaced during each generation), no scaling window, and an elitist selection strategy (the organism with the best performance survives intact into the next generation).
The random floating point numbers 0 < R < 1 used for selecting energy differentials based on the annealing temperature R < exp( ; E T ), the crossover points, the mutation rates, and the selection of organisms are produced using the subtractive method algorithm [36].Finally, the evaluative function that we used for calculating E is based on the decomposition method [2].

Method comparison
Before detailing the comparative results of our examination, it is interesting to visualize the operation of the two stochastic methods.Figure 2 depicts the runtime behavior of the two methods.Each point on the two scatter charts represents a given production line throughput value at a specific step of the algorithm.Both charts depict the calculation of the placement of 30 buffers in a balanced line of 10 stations.The simulated annealing algorithm optimizes a single solution in the specific example in 45:000 iterations.The solution's throughput value oscillates as both better and worse solutions are randomly selected at each iteration step.
As can be seen on the chart, the oscillation width decreases following the algorithm's exponential cooling schedule and converges towards the optimal value.The genetic algorithm is based on the implicit parallelism of the solutions represented by the initial population depicted on the chart's z-axis.Thus, in the specific example, it terminates with an optimal configuration after 250 generations.As the chart demonstrates the search starts with a wide spectrum of different solutions which are evaluated and evolve in parallel with non-optimal solutions gradually becoming extinct.Mutations and recombinations regenerate suboptimal solutions, but, due to the "survival of the fittest" organism selection strategy, their survival does not last for long.Our first comparison experiment concerned the algorithm operation on balanced lines for cases where exact solutions were known.In Figure 3 we present the optimum throughput configurations for balanced lines found using the stochastic methods against the throughput found using complete (for 9 stations) and reduced enumeration techniques.It is apparent that the stochastic algorithm results are almost identical and follow closely the results obtained by the other methods.Both methods are subject to the reduced evaluative accuracy of the decomposition method compared to the Markovian model.In addition to the balanced line evaluation, we compared the stochastic methods against unbalanced line enumeration using the Markovian evaluative procedure for a variety of line sizes, service time configurations, and available buffer space.The results are summarized in Figure 4.It is apparent, that the stochastic method configurations -although identical to each other -are not always optimal for limited available buffer space; however, they quickly converge towards the optimal configurations as buffer space increases.This difference can be accounted by the use of the fast decomposition evaluative procedure used in the stochastic algorithm implementation yielding approximate results against the use of the Markovian evaluative procedure for the enumeration method yielding exact results.Our goal for using stochastic methods is to optimize large production line problems where the cost of other methods is prohibitively expensive.As an example the reduced enumeration method when run on a 15 station line with a buffer capacity of 30 units took more than 10 hours to complete on a 100MHz Pentium processor.As shown in Figure 5 the cost of the stochastic methods is higher than the cost of the full and reduced enumeration methods for small lines and buffer allocations.However, it quickly becomes competitive as the number of stations and the available buffer size increase.In addition, the performance of the genetic algorithm implementation is approximately an order of magnitude better than the simulated annealing implementation.Notice that -in contrast to the deterministic methods -the stochastic method cost does not increase together with the available buffer space and that it increases only linearly with the number of stations.Finally, Figure 6 depicts the comparative performance and calculated throughput for the two stochastic methods when optimizing lines of up to 400 stations and 1200 buffers.The genetic algorithm implementation producing solutions with only 2:000:000 evaluations even for 400 station lines is clearly the performance winner.However, as depicted on the right hand chart, the throughput of the line configuration found by the genetic algorithm is consistently lower than the throughput of the line found by the simulated annealing method.The results we obtained could not be independently verified, because no other numerical results for the buffer allocation problem in large production lines can be found in the open literature.

4 .
Create new line with a random redistribution of buffer space.]Move R n space from a source buffer B Rs to a destination buffer B R d : set B 0 B, set R s b rand 2 : : : K +1)c, set R d b rand 2 : : : K + 1)c, set R n b rand 0 : : : B Rs + 1 ) c, set B Rs B Rs ; R n , set B R d B R d + R n . 5. Calculate energy differential.]Set E O K (B) ; O K (B 0 ).

6 . 7 .
Decide acceptance of new configuration.]Accept all new configurations that are more efficient and, following the Boltzmann probability distribution, some that are less efficient: if E < 0 or exp( ; E T ) > rand(0 : : : 1), set B B 0 , set U U + 1 .Repeat for current temperature].Set S S + 1 .If S < maximum number of steps, go to step 4.

3 .
Create roulette selection probability vectorR .] Set R i

Figure 3 :
Figure 3: Computed throughput of simulated annealing S(SA, Deco) and genetic algorithms S(GA, Deco) compared with complete enumerations using the exact S(CE, Exact) and the decomposition evaluative methods S(CE, Deco) for 9 stations (left); compared with reduced enumeration S(RE, Deco) for 15 stations (right).

Figure 5 :
Figure 5: Performance of simulated annealing S(SA, Deco) and genetic algorithms S(GA, Deco) compared with complete S(CE, Deco) and reduced S(RE, Deco) enumerations for 9 stations and 15 stations.Note the l o g 10 scale on the ordinate axis.

Figure 6 :
Figure 6: Performance and accuracy of simulated annealing S(SA, Deco) compared with genetic algorithms S(GA, Deco) for large production lines.Note the l o g 10 scale on the ordinate axis.