Fusion Global-Local-Topology Particle Swarm Optimization for Global Optimization Problems

In recent years, particle swarm optimization (PSO) has been extensively applied in various optimization problems because of its structural and implementation simplicity. However, the PSO can sometimes find local optima or exhibit slow convergence speed when solving complexmultimodal problems. To address these issues, an improved PSO scheme called fusion global-local-topology particle swarm optimization (FGLT-PSO) is proposed in this study. The algorithm employs both global and local topologies in PSO to jump out of the local optima. FGLT-PSO is evaluated using twenty (20) unimodal and multimodal nonlinear benchmark functions and its performance is compared with several well-known PSO algorithms. The experimental results showed that the proposed method improves the performance of PSO algorithm in terms of solution accuracy and convergence speed.


Introduction
PSO is a population-based metaheuristic algorithm introduced by Kennedy and Eberhart [1] in 1995.The algorithm imitates the social behavior of bird flocking or fish schooling to find the global best solution.Due to the simple concept, having a few parameters and being easy to implement, PSO has received much more attention to solve real-world optimization problems [2][3][4][5][6] in recent years.Nevertheless, PSO may easily get trapped in local optima when solving complex multimodal problems [7].Hence, a number of variant PSO algorithms have been proposed in the literature to avoid the local optima and to find the best solution promptly.
The algorithm applies two different topologies to find a good solution: global and local topologies.In global topology, the position of each particle is affected by the best-fitness particles of the entire population in the search space while each particle is influenced by the best-fitness particles of its neighborhood in the local topology.Kennedy and Mendes proposed local (ring) topological structure PSO (LPSO) [8] and the Von Neumann topological structure PSO (VPSO) [9].Mendes et al. [10] introduced the fully informed particle swarm (FIPS) algorithm and Ratnaweera et al. [11] suggested self-organizing hierarchical particle swarm optimizer with time-varying acceleration coefficients (HPSO-TVAC).Other researchers presented the several variants of PSO algorithms such as dynamic multiswarm PSO (DMS-PSO) [12], comprehensive learning PSO (CLPSO) [13], median-oriented particle swarm optimization (MPSO) [14], centripetal accelerated particle swarm optimization (CAPSO) [15], quadratic interpolation PSO (QIPSO) [16], quantum-behaved particle swarm optimization (QPSO) [17], and adaptive particle swarm optimization (APSO) [18].
Although the aforementioned algorithms have obtained satisfactory results, there are still some disadvantages in their utilization.For example, LPSO presents a slow convergence rate in unimodal functions [14,15] or CLPSO is not good for solving unimodal problems [13].Moreover, some of the algorithms have a better performance than PSO but their structures are not as simple as PSO.
To overcome the disadvantages, this study introduces fusion global-local-topology particle swarm optimization (FGLT-PSO).The proposed algorithm performs a global search over the entire search space with a fast convergence speed using hybridizing two local and global topologies in PSO to jump out from local optima.
The remainder of this paper is organized as follows.In Section 2, a brief review of PSO is provided followed by some well-known PSO algorithms.The proposed algorithm is described in Section 3 in detail.In Section 4, FGLT-PSO is used to solve several benchmark functions and its performance is compared with the other PSO algorithms in the literature.Finally, conclusions and the future research directions are presented in Section 5.

Particle Swarm Optimization (PSO)
2.1.PSO Framework.The PSO algorithm is a populationbased metaheuristic algorithm that applies two approaches of global exploration and local exploitation to find the optimum solution.The exploration is the ability of expanding search space, where the exploitation is the ability of finding the optima around a good solution.The algorithm is initialized by creating a swarm, that is, population of particles, with random positions.Every particle is shown as a vector ( ⃗   , ⃗   , ⃗   ) in a -dimensional search space where ⃗   and ⃗   are the position and velocity, respectively, and ⃗   is the personal best position ( best ) found by the th particle: 1   ,  2  , . . .,    ) for  = 1, 2, . . ., , , V 2  , . . ., V   ) for  = 1, 2, . . ., , ⃗   = ( 1   ,  2  , . . .,    ) for  = 1, 2, . . ., . ( In addition, the best position obtained by the entire population ( ⃗ ) is computed to update the particle velocity: 1   ,  2  , . . .,    ) . ( Based on ⃗   and ⃗ , the next velocity and position of the th particle are computed using (3) and (4) as follows: ( + 1) =    () + V   ( + 1) , where V   ( + 1) and V   () are the next and current velocity of the th particle, respectively. is inertia weight,  1 and  2 are acceleration coefficients, and rand 1 and rand 2 are random numbers in the interval [0, 1]. is the number of particles;    ( + 1) and    () are the next and current position of the th particle.
Also, |V   ( + 1)| < V max and V max is set to a constant bounded based on the search space bound.A larger value of w encourages global exploration (searching new areas), while a smaller value provides a local exploitation.
In (3), the second and the third terms are called cognition and social term, respectively.The two models applied to choose ⃗  are known as best (for global topology) and best (for local topology) models.In this paper, the best model and best model are called PSO and LPSO, respectively.
LPSO [8] and VPSO [9] were proposed based on a local topology to avoid premature convergence rate in solving multimodal problems.FIPS algorithm [10] is another PSO algorithm which uses the information of the entire neighborhood to guide the particles for finding the best solution.Dynamic multiswarm PSO (DMS-PSO) [12] was suggested by Liang and Suganthan to dynamically enhance the topological structure.Ratnaweera et al. [11] proposed HPSO-TVAC algorithm based on linearly time-varying acceleration coefficients where a larger  1 and a smaller  2 are set at the beginning and gradually reversed throughout the search.Liang et al. [13] presented comprehensive learning particle swarm optimization (CLPSO) which focused on avoiding the local optima by encouraging each particle to learn its behavior from other particles on different dimensions.
In another research, a selection operator for PSO was first introduced by Angeline [30].It is similar to what was used in a genetic algorithm (GA).Other researchers used a part of crossover [31] and mutation [29] operations from GA into PSO.Pant et al. proposed a quadratic crossover operator to PSO algorithm called quadratic interpolation PSO (QIPSO) [16].An adaptive fuzzy particle swarm optimization (AFPSO) [19] was proposed to utilize fuzzy inferences for adjusting acceleration coefficients.Meanwhile, the quadratic crossover operator [16] was used in the proposed AFPSO algorithm (AFPSO-QI) [19] to have better performance in solving multimodal problems.Zhan et al. presented an adaptive particle swarm optimization (APSO) [18] using a real-time evolutionary state estimation procedure and an elitist learning strategy.A variant of PSO algorithm based on orthogonal learning strategy (OLPSO) [32] was introduced to guide particles for discovering useful information from their personal best positions and from their neighborhood's best position in order to fly in better directions.Gao et al. [33] used PSO with chaotic opposition-based population initialization and stochastic search technique to solve complex multimodal problems.The algorithm called CSPSO finds new solutions in the neighborhoods of the previous best positions in order to escape from local optima in multimodal functions.Beheshti et al. proposed median-oriented particle swarm optimization (MPSO) [14] and centripetal accelerated particle swarm optimization (CAPSO) [15] based on Newton's laws of motion to accelerate the learning and convergence of optimization problems.

FGLT-PSO: The Proposed Method
3.1.FGLT-PSO Algorithm.FGLT-PSO tends to overcome the disadvantages of PSO by avoiding local optima and accelerating convergence speed.According to [14,15], PSO has shown a better performance than LPSO in unimodal problems and LPSO illustrates good results in multimodal problems.Hence, both local and global topologies are hybridized in FGLT-PSO to increase the convergence rate and to avoid trapping into local optima.
In FGLT-PSO algorithm, each particle uses the best position found by its neighbors ( ⃗  best ) to update the particles' velocities: The next position of each particle is computed based on the current position,    (), the next velocity, V   ( + 1), and the best position found by the swarm, ⃗  best , as follows: In (6),  is computed as Also,  1 (),  2 (), and  3 () are acceleration coefficients and modified according to (10): where  and  are the current iteration and the number of maximum iterations, respectively.
The second term in ( 6) is called the cognition term, and the third terms in ( 6) and ( 7) are named the social terms.In (7), |   ( + 1)| <  max and  max is set to a constant based on the search space bound.

Analysis of FGLT-PSO.
A metaheuristic algorithm explores new spaces to avoid trapping in a local optimum in the initial steps.Due to the poor exploration in the standard PSO (PSO), it can sometimes find local optima in multimodal problems.Sometimes, if a particle falls into a local optimum, it will not be able to get out of it.That is, if ⃗  best obtained through the population lies in a local optimum while the current position and the personal best position of particle  are in the same local optimum, the second and third terms of (3) tend to zero and  decreases linearly to near zero.Consequently, the next velocity of particle  tends to zero, and its next position in (4) does not change; thus, the particle remains in the local optimum.Hence, the main aim in FGLT-PSO is to overcome the poor exploration and to increase the convergence rate by combining the local and global searches as shown in Figure 1.The particles move in the search space based on the best solutions found by their neighbors ( ⃗  best ) and the swarm ( ⃗  best ).At the beginning, the particles search new spaces.By lapse of iterations, the exploration should fade out and the exploitation should fade in.It means the particles accelerate to the good solution and make search around it to find the best solution.

Experimental Results
In this section, the FGLT-PSO algorithm is compared with some well-known PSO algorithms.The algorithms are tested using various unimodal and multimodal functions in different dimensions.Several benchmark functions [34,35] are selected to evaluate the performance of proposed method.In unimodal functions, the convergence rate of search algorithm is more interesting than the final results because other methods have been designed to optimize these kinds of functions.In multimodal functions, finding an optimal (or a good near-global optimal) solution is important.These functions are more difficult to optimize because the number of local optima exponentially increases as the dimension increases.Therefore, the search algorithms should not become trapped in a local optimum and should be able to obtain good solutions.
The rotation of function increases the function complexity.It does not affect the shape of function.The variable ⃗  is computed using an orthogonal matrix  [36] and applied to obtain the fitness value of rotated function as follows: In shifted functions, the global optimum ⃗  * = ( * 1 ,  * 2 , . . .,  *  ) is shifted to the new position ⃗  = ( 1 ,  2 , . . .,   ).All the test functions are shown as follows.

Results of FGLT-PSO.
The results of FGLT-PSO are provided in three sections.In Section 4.2.1, the acceleration coefficients  1 ,  2 , and  3 in the proposed method are changed according to (10) and in Section 4.2.2, these factors are constant.In these sections, FGLT-PSO is evaluated using the benchmark functions with dimensions 10, 30, and 50.The number of maximum iterations is set at 5000 for  = 10, at 10000 for  = 30, and at 15000 for  = 50.The population size is set to 50 ( = 50).Also,  decreases linearly from 0.9 to 0.4.In Section 4.2.3, the results of FGLT-PSO are compared with those of several well-known PSO algorithms from [19] on the common functions.In this section, the population size is set to 30 ( = 30),  is 30, and the number of maximum iterations is set at 10000.
The ring topology is used as the neighborhood structure in the best model for the FGLT-PSO and LPSO algorithms and the number of neighbours for each particle is three.The algorithms are run independently times for the benchmark functions and the results are averaged.Moreover, Wilcoxon's rank sum test [37] is conducted in order to determine whether the results obtained by the FGLT-PSO are different from those generated by other algorithms with a statistical significance.The tests are shown in Tables 4, 5, 8, 9, 12, and 13, where ℎ-value = 1 indicates the case in which proposed algorithm significantly outperformed the compared algorithm with 95% certainty, ℎ-value = −1 represents that the compared algorithm is significantly better than the proposed algorithm, and ℎ-value = 0 denotes that the results of the two considered algorithms are not significantly different.In these tables, rows 1 (better), 0 (same), and −1 (worse) give the number of functions that the FGLT-PSO performs significantly better than, almost the same as, and significantly worse than the compared algorithm, respectively.
The acceleration coefficients  1 ,  2 , and  3 are updated based on (10).Their minimum and maximum values are as follows:  1 min = 0.5,  1 max = 2,  2 min = 1,  2 max = 2,  3 min = 0.5, and  3 max = 1.5.In these tables, the benchmark functions are divided to two categories: (1) unimodal and multimodal functions and (2) rotated, shifted, and shifted-rotated unimodal and multimodal functions.The experimental results demonstrate that FGLT-PSO performs superior results for most of the functions in all tested dimensions.
Tables 2 and 3 show the experimental results for all benchmark functions with dimension  = 10.As illustrated, the FGLT-PSO algorithm surpasses the PSO, LPSO, and QIPSO algorithms in minimizing functions (1)-( 6), ( 10), ( 12), ( 13), ( 15)- (18), and (20).Moreover, the proposed method provides significant improvements in functions (1), ( 2), ( 3), ( 10), ( 13), (17), and (18).In these functions, the convergent results attain the optimal (or good near optimal) solutions.In Tables 2 and 3, the average iteration for finding the best solution is computed.The average iteration is the required iterations to find the best solution by each algorithm.As shown, FGLT-PSO finds the best solutions faster than the other algorithms in the majority of functions.Also, it is noticeable that the FGLT-PSO algorithm achieves the best solution in a considerably lower iteration in functions (7) and (8).In these functions, the algorithms show identical results.
In addition, it is considerable that the PSO, LPSO, and QIPSO algorithms return the results far from the global optima as the dimension increases.This problem is clear in the functions (3), ( 5), ( 9), ( 10), ( 16), ( 17), (18), and (20) in Tables 10 and 11 with  = 50.These all indicate that   the proposed algorithm, FGLT-PSO, is more powerful and robust than the others for solving unimodal and multimodal functions.

The Results of Proposed Method with Constant Acceleration Coefficients.
In this section, the  1 ,  2 , and  3 as acceleration coefficients are set at constant values to compare with the presented results in the Section 4.2.1.The coefficient of cognition term ( 1 ) and social terms ( 2 and  3 ) are considered as  1 = 2 and  2 =  3 = 1.Table 14 shows the results of proposed method for the benchmark functions with dimensions 10, 30, and 50.As seen, the FGLT-PSO with the constant acceleration coefficients performs well in most of the functions.As the dimension increases, the FGLT-PSO with the variable acceleration coefficients (Section 4.2.1)shows the better performance than the constant one for functions (1), (2), (3), ( 5), (17), and (20).Also, the FGLT-PSO with the constant acceleration coefficients presents a better performance for functions (7), (13), and (18).

Comparison with the Other PSO Algorithms.
In this section, several well-known PSO algorithms are selected to assess the performance of proposed algorithm for the benchmarks.The PSO, QIPSO, FIPS, DMS-PSO, CLPSO, AFPSO, and AFPSO-QI algorithms are considered for the comparison.The details of these algorithms are listed in Table 15.The FGLT-PSO is run 30 times and the average best  solutions and the SD of results for eight common multimodal benchmark functions are compared with the reported results by [19] as illustrated in Table 16.The maximum iteration is 10000,  = 30, and  = 30.As seen, the FGLT-PSO provides better results than the other algorithms for the majority of functions (functions (11), ( 12), ( 13), ( 14), (17), and (20)) and has the first rank.

Conclusions
In this study, a fusion global-local-topology PSO algorithm (FGLT-PSO) has been presented to extend the search capability and to improve convergent efficiency by combining local and global topologies.The algorithm is a global search algorithm with several advantages.The benefits of algorithm can be summarized as the following: FGLT-PSO has a simple concept and structure; it is easy to implement and is not sensitive to increase of the dimension.A set of standard benchmarks, including unimodal, multimodal, rotated, shifted, and shifted-rotated unimodal and multimodal functions, have been used to evaluate the proposed algorithm.The average best results obtained by the FGLT-PSO have been compared with PSO, LPSO, QIPSO, FIPS, DMS-PSO, CLPSO, AFPSO, and AFPSO-QI.The experimental results show that the proposed FGLT-PSO algorithm enhances the accuracy of results compared with the other algorithms.

Figure 1 :
Figure 1: Graphical representation of the particle movement using FGLT-PSO algorithm.

Table 1 :
Dimensions, ranges, and global optimum values of test functions used in the experiments.

Table 3 :
Minimization results for the rotated and shifted unimodal and multimodal functions (maximum iteration = 5000 and  = 10).

Table 5 :
Comparison of FGLT-PSO with PSO, LPSO, and QIPSO for the rotated and shifted unimodal and multimodal functions using Wilcoxon's rank sum test ( = 10).

Table 7 :
Minimization results for the rotated and shifted unimodal and multimodal functions (maximum iteration = 10000 and  = 30).

Table 9 :
Comparison of FGLT-PSO with PSO, LPSO, and QIPSO for the rotated and shifted unimodal and multimodal functions using Wilcoxon's rank sum test ( = 30).

Table 11 :
Minimization results for the rotated and shifted unimodal and multimodal functions (maximum iteration = 15000 and  = 50).The results in this figure illustrate that FGLT-PSO tends to find the global optimum in  1 and  17 faster than PSO, LPSO, and QIPSO and obtains the highest accuracy for these functions from among all the algorithms.

Table 13 :
Comparison of FGLT-PSO with PSO, LPSO, and QIPSO for the rotated and shifted unimodal and multimodal functions using Wilcoxon's rank sum test ( = 50).

Table 15 :
Some well-known PSO algorithms in the literature.