A Local and Global Search Combined Particle Swarm Optimization Algorithm and Its Convergence Analysis

Particle swarm optimization algorithm (PSOA) is an advantage optimization tool. However, it has a tendency to get stuck in a near optimal solution especially for middle and large size problems and it is difficult to improve solution accuracy by fine-tuning parameters. According to the insufficiency, this paper researches the local and global search combine particle swarm algorithm (LGSCPSOA), and its convergence and obtains its convergence qualification. At the same time, it is tested with a set of 8 benchmark continuous functions and compared their optimization results with original particle swarm algorithm (OPSOA). Experimental results indicate that the LGSCPSOA improves the search performance especially on the middle and large size benchmark functions significantly.


Introduction
Swarm intelligence (SI) techniques, including particle swarm optimization (PSO) and ant colony optimization (ACO), use swarm behavior to solve the problem, and they use the concept of neighborhood intelligence along with individual intelligence.Particle swarm was originally formulated to study the interesting concept of a social behavior (bird flocking or fish schooling) in a simplified manner in simulation.However, very soon the potential of this technique was realized to develop into a powerful optimization tool which can be successfully applied in the fields of both continuous and discrete optimization problems such as function optimization, scheduling and hence received many attentions since it is originated.
The particle swarm concept originated as a simulation of simplified social system, and this technique has been developing rapidly.Eberhart and Kennedy in [1,2] originally formulated particle swarm in 1995, and recently, several investigations have been undertaken to improve the performance of original PSO.There are many improved PSO algorithm such as Shi and Eberhart who introduced an inertia weight making the influence of the previous velocity on the new velocity in [3][4][5] and Kennedy and Mendes use cluster analysis improving PSO's performance in [6].Angeline presented the model of improved PSO algorithm called (HPSO) in [7] which uses selection operation of evolution computing.Literature [8] Suganthan sets up the model of PSO with adjacent filed operation.Jiao et al. present a dynamic inertia weight particle swarm optimization algorithm in paper [9] in which the dynamic inertia weight that decreases according to iterative generation increasing is used.Many experts also studied others properties of PSO such as Clerc and Kennedy in [10,11] who have researched the particle swarm explosion stability and convergence in a multidimensional complex space.Eberhart and Shi have investigated PSO developments, applications, and resources in [12] and parameter selection in [13] and dimension selection methods in [13].
PSO has been successfully applied in many areas: function optimization, artificial neural network training, fuzzy system control, and other areas where GA can be applied.Parsopoulos and Vrahatis approach global optimization problems using PSO in [14,15] and Abido uses PSO optimizing power flow in [16].PSO technique have been used in assignment problem in [17], reactive power and voltage control considering voltage security assessment in [18], and so forth.
PSO algorithm is widely used, such as the paper [19,20] that introduces a Hybrid Particle Swarm algorithm with artificial Immune Learning (HPSIL) for solving Fixed Charge Transportation Problem (FCTP).Hamta et al. in paper [21] presented a hybrid PSO algorithm for a multiobjective assembly line balancing problem with flexible operation times, sequence-dependent setup times, and learning effect.Chou [22] used a particle swarm optimization with cocktail decoding method for hybrid flow shop scheduling problems with multiprocessor tasks.This study searches for a number of solid decoding methods that can be incorporated into the cocktail decoding method.Then, it develops a particle swarm optimization (PSO) algorithm that can be combined with the cocktail decoding method.PSO has particularly gained prominence due to its relative ease of operation and capability to quickly arrive at an optimal/near-optimal solution.However it was pointed out in our studies that original PSO might frequently tend to get stuck in a near optimal solution in reaching optimum solutions especially for middle or large size function optimization problems.According to the shortcoming of original PSO, a local and global search combine particle swarm optimization algorithm is presented in this paper, and the simulation results show that it is efficacious.At the same time, this paper rigorously analyzes new PSO algorithm convergence and obtains its convergence qualification.
This work differs from the existing ones at least in four aspects: firstly, it proposes a local and global search combine particle swarm optimization (LGSCPSO) algorithm iterative formula, in which all particles share the best information of the local particles, global particles and neighborhood particles.Secondly, it finds the best combine parameters of LGSCPSO for different size optimization problems.Thirdly, it is to compare the original PSO with LGSCPSO and shows that the latter is more efficacious for optimization problems.Fourthly, it strictly analyzes the PSO algorithm's convergence and obtains its convergence qualification.The rest of the paper is organized as follows: The next section introduces the original PSO model.The iteration formulation of LGSCPSO algorithm is presented in Section 3. In Section 4, it analyzes the PSO algorithm's convergence and finds its convergence qualification.Then, it describes the test functions, experimental settings, and compares experimental results of original PSO with LGSCPSO algorithm.Finally, Section 5 summarizes the contribution of this paper and conclusions.

Original Particle Swarm
Optimization Algorithm

Description of Original Particle Swarm Optimization
Algorithm.Particle swarm optimization (PSO) is an evolutionary computation technique, and its concept originated as a simulation of a simplified social system.The major difference is that the evolution computing techniques use genetic operators whereas swarm intelligence techniques use the physical movements of the individuals in the swarm.PSO is distinctly different from other evolutionary-type methods in a way that it does not need complex encoding and decoding processes and does not use the operation (such as crossover and/or mutation), and it takes real numbers as particles in the aspects of representation solution.From the procedure, one can learn that PSO shares many common points with GA; for example, they start with a neighborhood of a randomly generated population, evaluate the population with fitness values, update the population and search for the optimum with random techniques, and do not guarantee optimal.Similar to other population-based algorithms, PSO as an optimization tool can solve a variety of difficult optimization problems.Compared to GA, one of the important attractive factors of PSO is simple that there are very few parameters to adjust.It can achieve the optimal or near-optimal solutions in a rather short time without enormous iterative computations in digital implementation.Swarm intelligence simulates a social behavior such as bird flocking to a promising position for certain objectives in a multidimensional space.It is initialized with a population of random solutions and searches for optima by updating generations.PSO system combines local search method (through self-experience) with global search methods (through neighboring experience), attempting to balance exploration.Like evolutionary algorithm, PSO search uses a population (called swarm) of individuals (called particles) that are updated from iteration to iteration.Each particle represents a candidate position (i.e., solution) to the problem at hand, resembling the chromosome of GA.A particle is treated as a point in an M-dimension space, and the status of a particle is characterized by its position and velocity.Initialized with a swarm of random particles, PSO is achieved through particle flying along the trajectory that will be adjusted based on the best experience or position of the one particle (called local best) and ever found by all particles (called global best).PSO updates a population of particles with the internal velocity and attempts to profit from the discoveries of themselves and previous experiences of all other companions.

The Model of Original Particle Swarm Optimization
Algorithm.In every search iteration, each particle is updated by following two "best" values.The first one is the best solution (fitness) it has achieved so far, and this value is called Best.Another "best" value obtained so far by any particle in the population, and this best value is a global best and called Best.After finding the two best values, the particle updates its velocity and positions with following formulas (1) and (2): ( + 1) =   () + V  ( + 1) ( = 1, 2, . . ., ;  = 1, 2, . . ., ) . ( In (1) The main disadvantage of the above original PSO is that it is difficult to keep the diversity of population and to balance local and global search and hence it may result in local optimal solutions.Besides, their search rates are commonly lower and sometimes need more computation when solving some difficult optimization problems.The original PSO frequently gets into the local solution, especially when the problem size is middle or large.In the following a local and global search combine particle swarm optimization algorithm is proposed to deal with the above disadvantages of original PSO in solving optimization problems.

A Local and Global Search Combined
Particle Swarm Optimization Algorithm LGSCPSO algorithm improves upon the original PSO variation to increase accuracy of solution without sacrificing the speed of solution significantly, and its detailed information will be given in following.Suppose that the searching space is D-dimensional and m particles form the colony.The th particle represents a Ddimensional vector   = ( 1 ,  2 , . . .,   ) ( = 1, 2, . . ., ).It means that the th particle locates at   in the searching space and the position of each particle is a potential solution.We could calculate the particle's fitness by putting its position into a designated objective function.When the fitness is lower, the corresponding   is "better." The th particle's "flying" velocity is also a D-dimensional vector, denoted as   = (V 1 , V 2 , . . ., V  ).Denote the best position of the th particle as   = ( 1 ,  2 , . . .,   ), the best position of the local as   = ( 1 ,  2 , . . .,   ), and the best position of the global as   = ( 1 ,  2 , . . .,   ), respectively.After finding the three best values, the particle of LGSCPSO updates its velocity and positions with the following formulas: where  is an inertia weight, which is initialized typically in the range of [0, 1].The neighborhood particles of  generation are composed by the best neighborhood particle (local the particle) of  generation. ∈ [0, 1] is weight index that is chosen according to different optimization problem, which reflects relatively important degree of the best position of the th particle and the best position of the th generation neighborhood particle.The variables  1 ,  2 are acceleration constant, which control how far a particle will move in a single iteration, and others parameters are same as in ( 1) and ( 2).The searching is a repeat process, and the stop criteria are that the maximum iteration number is reached or the minimum error condition is satisfied.The stop condition depends on the problem to be optimized.In LGSCPSO algorithm, each particle of the swarm shares mutual information globally and benefits from discoveries and previous experiences of all other colleagues during the search process.

Pseudocode of LGSCPSO (See Pseudocode 1).
Because PSO has very deep intelligent background, it is suitable for science computation and engineering applications.A thorough mathematical foundation for the methodology was not developed so far with the step of the algorithm; however, it has been proven to be very effective for application.

Convergence Analysis of LGSCPSO.
In this section the LGSCPSO algorithm's convergence is studied.Theorem 1.In LGSCPSO algorithm in which its recurrence equation is (3) and (4), when parameters relation 3) and ( 4) we have From ( 4) we can get V  () =   ()−  (−1), and combining (5) we have LGSCPSO algorithm's recurrence equation is linearity constant coefficient nonhomogeneous, and the secular equation of corresponding homogeneous recurrence equation is as follows: Latent roots of above secular equation are as follows: According to the relations of recurrence equation and its special solution, let the special solution of LGSCPSO algorithm's recurrence equation ( 6)  * () =  substitute into (6), and we can solve According to the relations of recurrence equation's general solution, special solution, and its latent roots, we can obtain the general solution of LGSCPSO algorithm's recurrence equation ( 6) as follows: The initial solution and velocity of LGSCPSO algorithm recurrence equation is stochastically generated; suppose first and second solution is   (1) and   (2), respectively, and according to (10), we can obtain that To use (11) substitution into (10) simplification, we can obtain the general solution of LGSCPSO recurrence equation ( 6) as follows: If the limit of above () exists when  → ∞, the LGSCPSO algorithm is convergent.Let So the Δ of above function is as follows: Evidently, when hold, we can obtain the function (13 From LGSCPSO algorithm theory, we obviously get that   () and   () are descending with  increase, and  is also descending with  increase. When Because is monotone decreasing and have lower bound which is its optimum solution, lim exist, namely, the endless particle sequence brought about by LGSCPSO algorithm's recurrence equation is limited.So we can obtain that the LGSCPSO algorithm is convergent when its parameters relations 0

Numerical Simulation
where They can be divided into unimodal (function  1 - 3 ) and multimodal functions (function  4 - 8 ), where the number of local minima increases exponentially with the problem dimension.each test function are listed in Table 1.To get the average performance of the LGSCPSO algorithm ten runs on each problem instance were performed and the solution quality was averaged.The comparisons results of LGSCPSO and Original PSO algorithm are shown in Table 1 of the appendix.From 1 one can observe that  ∈ [0.2, 0.4] have the highest performance since using them has smaller minimum and smaller arithmetic mean in relation to the solutions obtained by the others, especially  ≈ 0.4 has the better search efficiency.In LGSCPSO algorithm, various parameter  should be selected for optimizing different problems, as a whole, it is better when  ∈ [0.3 0.9], especially  ≈ 0.4.From simulation results we can obtain that the LGSCPSO algorithm is clearly better than the original PSO for continuous nonlinear function optimization problem.The convergence figures of most effective LGSCPSO comparing with original PSO for 8 instances are as follows.

Experimental Results and Comparison. The experimental results of original PSO and LGSCPSO algorithm on
From Figure 1 we can discover that the convergence rate of LGSCPSO algorithm is clearly faster than the OPSO on every benchmark function, especially it is more efficacious than OPSO for middle and large size optimization problem.Accordingly, we can state that the LGSCPSO algorithm is more effective than OPSO algorithm.For test function 5, the search capability of LGSCPSO algorithm is worse than that of OPSO at the beginning of the iteration.However, seen from the enlarged figure, the search capability of LGSCPSO is markedly enhanced along with the iteration increasing.In addition we can find that the simulation data from test function 5 has a strong advantage.

Conclusions and Perspectives
According to the shortcoming of OPSO algorithm that it frequently gets the local solution especially solving the middle or large size problem, a LGSCPSO algorithm is presented, and simulations show that it is efficacious.The performance of the new approach is evaluated in comparison to OPSO algorithm for eight representative instances with different dimensions and obtained results show that LGSCPSO algorithm is efficacious for solving optimization problems.
The proposed LGSCPSO algorithm approach in this paper can be considered as effective mechanisms from this point of view.There are a number of research directions that can be considered as useful extensions of this research.Although the proposed algorithm is tested with eight representative instances, a more comprehensive computational study should be made to test the efficiency of proposed solution technique.In the future it is maybe used for solving other discrete combinatorial optimization problems such as traveling salesman problem and scheduling.
,   = ( 1 ,  2 , . . .,   ) is the best previous position of the th particle (also known as Best).According to the different definitions, there are two different versions of PSO, with local version, and particles only have information of their own and their neighbors' bests, rather than that of the entire neighborhood.If   = ( 1 ,  2 , . . .,   ) is the best position among all the particles in the swarm (also known as Best), such a version is called the global version.If   is max ∼  which is a parameter specified by the user.If the iteration formulas would cause the velocity on that dimension to exceed  max ∼ , then the velocity on that dimension is limited to  max .The termination criterion for the iterations is determined according to whether the max generation or a designated value of the fitness of   is reached.
Step 1.Let parameters; including swarm size PS, maximum of generation endgen and others parameters will be used in LGSCPSO algorithm.Find new   ,   of the swarm and   of each particle by comparison, and update   and   ; (c) Update   using the best particle of  generation colony;} } Step 3. Output optimization results.

Table 1 :
The comparison results of the PSO algorithm and the LGSCPSO algorithm.

Table 1 ,
Fun and Dim denote function and its dimension, respectively; the Best is this function's optimum value; PS and EG indicate the population size and algorithm terminate generation.PSO and LGC denote the original PSO and LGSCPSO algorithm.The better solutions and the parameters found in LGSCPSO are illustrated with bold letters, and the best solutions are shown in data with asterisk.