The presented paper deals with the comparison of selected random updating strategies of inertia weight in particle swarm optimisation. Six versions of particle swarm optimization were analysed on 28 benchmark functions, prepared for the Special Session on Real-Parameter Single Objective Optimisation at CEC2013. The random components of tested inertia weight were generated from Beta distribution with different values of shape parameters. The best analysed PSO version is the multiswarm PSO, which combines two strategies of updating the inertia weight. The first is driven by the temporally varying shape parameters, while the second is based on random control of shape parameters of Beta distribution.
The particle swarm optimisation—PSO—is a popular heuristic optimisation algorithm developed by Kennedy and Eberhart [
Recently the PSO oriented research focuses on the development of new adaptation strategies, which avoid the premature convergence of particle population, or being trapped in local optima. For example the periodic changes of number of particles in population enhance the PSO performance [
The improvement the estimation of particle’s velocity is an essential task in PSO research. It was shown that the inertia weight—IW—helps to increase the overall PSO search performance [
The random adaptations of inertia weight play an important role in improving the PSO performance [
Besides the adaptation strategies of PSO parameters the special attention has to be put on development the multiswarm PSO [
The comparison study of 12 different migration strategies 6 on 36 optimisation problems is provided in [
The aim of the presented paper is to compare selected version of PSO. The tested single and multiswarm versions of particle swarm optimisation are based on modifications of inertia weight, which are related to the random component controlled by the Beta distribution.
The remaining part of paper is arranged as follows. The description of PSO provides details on standard PSO, the proposed random inertia weight strategies, and the description of tested multiswarm PSO. Results comment on the finding based on extensive 10 dimensional computational experiments. The article summarizes the main findings in Conclusions.
The standard PSO (sPSO) modifies the location of particle
The new location of particle is computed as
The sPSO is based on the velocity update with the linear decreasing inertia term
The velocity update formula is restricted by
The proposed inertia weight modifications are based on random numbers generated using the Beta distribution. The density of Beta distribution
The selected densities of Beta distribution.
One of the main advantages of Beta distribution is that it describes probability densities with various shapes on the interval
Table
Tested inertia weight updates based on Beta distribution.
PSO version | The weight update formula | Random component |
---|---|---|
RBld |
|
|
|
||
RBrr |
|
|
|
||
RBRa |
|
|
The RBrr inertia weight version applies randomly selected shape parameters
The RBRa is modification of original of logistic mapping [
The new proposed multiswarm PSO combines the search of four subswarms. This PSO version is marked as BrBl. The algorithm follows the principles of multiswarm algorithms [
The migration period is controlled by the simple rule, which increases the number of generations between two successive migrations. The migration iteration
The cooperative subswarms are formed of the three subswarms. Their cooperation is based on migration with migration rate
The selection of subswarm for emigration is controlled randomly. Note that with the probability
The second group of subswarms is formed from one elitistic swarm. This subswarm searches over the search space and receives the all best particles from cooperative swarms. The best particles substitute the randomly selected particles from elitistic subswarm. The elitistic swarm does not share the knowledge of global best particle with cooperative subswarms.
The proposed modifications of inertia weight strategies were applied on 28 CEC2013 benchmark minimization problems [
The search space for all CEC2013 benchmark functions was
The computations were made using the R statistical environment 3.0.2 [
The single PSO parameter settings were based on [
The PSO with proposed inertia weight strategies was compared with standard PSO (sPSO) and AMPSO2. The AMPSO2 uses the Beta distribution on adaptive mutation of the personal best particles and global best particle [
The parameter settings particle initializations of BrBl subswarms were those used in single PSO. The BrBl migration rate
We relate the description of balance between the exploration and exploitation to the evolution of the variances and fitness values of global best particles generated by the all 51 optimisation runs.
The variance of tested PSO versions was described using the standard deviations of differences between fitness values and median, which was obtained from 51 runs in given iteration. The results for the first 4000 iterations on 12 selected benchmark problems are shown in Figures
The normalized standard deviation and global best model of 51 runs for f1, f2, f5, and f6; SDEV is the standard deviation, GBEST fitness of global best particle. Note: all values are shifted due to the logarithmic transformation of
The normalized standard deviation and global best model of 51 runs for f11, f15, f17, and f21; SDEV is the standard deviation, GBEST fitness of global best particle. Note: all values are shifted due to the logarithmic transformation of
The normalized standard deviation and global best model of 51 runs for f22, f23, f25, and f27; SDEV is the standard deviation, GBEST fitness of global best particle. Note: all values are shifted due to the logarithmic transformation of
On unimodal problems f1–f5 and multimodal problem f17 the AMPSO2 and sPSO show clearly different patterns in the evolution of standard deviations than PSO versions with Beta distribution. The PSO versions with Beta distribution show the decrease of the variance of swarm particles, while the AMPSO2 and sPSO show the stagnation. These similar patterns of decrease and stagnation are apparent on the fitness values of global best particles.
Those patterns are connected to the convergence of tested PSO versions. For example on f1 problem all PSO versions based on Beta distribution found earlier the optimum than AMPSO2 and sPSO (see the results of Table
The BrBl shows the highest variances in the beginning of iteration search. These are connected to the intensive migrations, performed during the early stages of optimisation search. The main benefit is shown in later rapid decrease of fitness value (e.g., see the results in Figure
The BrBl version also shows the increase of variance during the search process on f15 and f23. This fact is again connected to the finding of better solutions in terms of values of global best particle (see Table
The minimum values achieved at
CEC problem | Min. | RBRa | RBrr | AMPSO2 | RBld | sPSO | BrBl |
---|---|---|---|---|---|---|---|
f1 | −1400 |
|
|
|
|
|
|
f2 | −1300 | 31802.55 |
|
204814.76 | 20573.61 | 51187.06 | 18609.16 |
f3 | −1200 | −1199.74 | −1199.95 | 17054.84 | −1199.82 |
|
−1199.95 |
f4 | −1100 | −964.04 | −947.26 | 576.38 |
|
−913.96 | −388.49 |
f5 | −1000 |
|
|
|
|
|
|
f6 | −900 | −899.93 | −899.99 | −899.51 | −899.81 | −899.76 |
|
f7 | −800 |
|
−799.32 | −798.57 | −799.92 | −799.95 | −798.90 |
f8 | −700 | −679.83 | −679.80 | −679.79 | −679.78 | −679.84 |
|
f9 | −600 | −599.25 | −599.23 | −599.25 | −598.43 |
|
−599.37 |
f10 | −500 | −499.93 |
|
−499.82 | −499.91 | −499.91 | −499.94 |
f11 | −400 |
|
|
−397.56 | −399.01 |
|
|
f12 | −300 |
|
−294.03 | −284.82 | −293.04 |
|
|
f13 | −200 | −193.73 | −193.73 | −182.54 | −193.04 |
|
−192.22 |
f14 | −100 |
|
|
−40.45 | −93.11 | −93.05 | −93.12 |
f15 | 100 |
|
382.90 | 624.16 | 343.59 | 463.56 | 295.58 |
f16 | 200 | 200.70 | 200.24 | 200.38 | 200.47 | 200.58 |
|
f17 | 300 | 303.29 | 301.60 | 324.08 | 302.45 | 304.34 |
|
f18 | 400 | 414.78 |
|
433.38 | 414.77 | 415.52 | 414.11 |
f19 | 500 |
|
500.07 | 500.89 | 500.19 | 500.31 | 500.14 |
f20 | 600 | 601.96 | 601.94 | 602.44 | 602.03 |
|
602.11 |
f21 | 700 | 800.00 | 800.00 | 800.03 | 800.00 |
|
800.00 |
f22 | 800 | 829.38 | 823.36 | 925.66 | 829.06 | 830.92 |
|
f23 | 900 | 1149.35 |
|
1613.80 | 1127.72 | 1181.06 | 1338.13 |
f24 | 1000 |
|
1114.66 | 1125.87 | 1112.24 | 1136.31 | 1109.72 |
f25 | 1100 | 1301.93 | 1303.37 | 1302.83 | 1302.10 | 1301.53 |
|
f26 | 1200 | 1307.96 | 1305.97 | 1321.06 | 1306.97 | 1308.95 |
|
f27 | 1300 | 1615.16 | 1622.81 | 1658.37 | 1636.13 | 1607.02 |
|
f28 | 1400 |
|
|
1500.01 |
|
|
|
The minimum values achieved at
CEC problem | Min. | RBRa | RBrr | AMPSO2 | RBld | sPSO | BrBl |
---|---|---|---|---|---|---|---|
f1 | −1400 |
|
|
−1147.93 |
|
−1265.72 |
|
f2 | −1300 | 222040.73 | 322300.27 | 1498763.45 |
|
3959345.31 | 236444.51 |
f3 | −1200 | 2136685.60 | 392088.96 | 1193329345.61 |
|
331615655.13 | 6232942.40 |
f4 | −1100 | 5620.73 | 5057.65 | 6101.20 |
|
7088.07 | 5451.37 |
f5 | −1000 | −999.98 |
|
−943.64 |
|
−961.61 | −999.97 |
f6 | −900 | −899.70 | −899.84 | −878.23 | −899.57 | −881.82 |
|
f7 | −800 | −795.95 |
|
−752.51 | −794.56 | −771.83 | −788.15 |
f8 | −700 | −679.73 | −679.75 | −679.75 | −679.78 |
|
−679.73 |
f9 | −600 | −598.53 | −597.77 | −593.72 | −597.54 | −593.42 |
|
f10 | −500 | −498.94 |
|
−464.60 | −499.36 | −468.55 | −498.81 |
f11 | −400 | −394.23 | −395.61 | −368.60 | −395.74 | −362.84 |
|
f12 | −300 | −283.02 |
|
−252.46 | −292.11 | −258.93 | −286.85 |
f13 | −200 | −182.06 | −183.29 | −147.10 |
|
−148.35 | −179.85 |
f14 | −100 |
|
116.03 | 199.00 | 73.64 | 1221.41 | 25.31 |
f15 | 100 | 1366.48 | 859.64 | 1416.16 | 834.71 | 1331.53 |
|
f16 | 200 | 201.00 | 200.58 | 200.73 | 200.87 | 200.89 |
|
f17 | 300 | 327.16 | 322.55 | 366.99 | 323.63 | 365.88 |
|
f18 | 400 | 446.89 | 438.03 | 473.89 | 435.77 | 482.47 |
|
f19 | 500 | 501.16 | 501.02 | 508.15 | 500.75 | 505.62 |
|
f20 | 600 | 602.94 |
|
603.07 | 602.86 | 603.40 | 602.98 |
f21 | 700 | 803.41 | 800.72 | 1080.72 |
|
1053.29 | 801.40 |
f22 | 800 | 1107.86 | 1009.20 | 1318.77 |
|
2022.70 | 1003.78 |
f23 | 900 | 1580.41 | 1796.86 | 2286.08 | 1740.24 | 2546.45 |
|
f24 | 1000 | 1137.37 | 1144.17 | 1175.66 |
|
1172.10 | 1135.56 |
f25 | 1100 | 1305.98 | 1305.48 | 1315.70 | 1303.40 | 1319.22 |
|
f26 | 1200 | 1321.41 | 1321.35 | 1347.29 | 1311.11 | 1357.60 |
|
f27 | 1300 | 1646.86 | 1641.18 | 1799.34 | 1659.35 | 1776.92 |
|
f28 | 1400 | 1503.26 | 1500.31 | 1692.99 |
|
1667.17 | 1500.63 |
Following the recommendation of CEC2013 the maximum function evaluation (FES) was set as
The PSO versions with Beta distribution components show the best convergence properties on all benchmark problems for FES
The results of FES
The comparison of mean performance of all 51 runs for FES
The contrast test on best values achieved on
RBRa | RBrr | AMPSO2 | RBld | sPSO | BrBl | Ranking | |
---|---|---|---|---|---|---|---|
RBRa | — |
|
|
|
|
|
4 |
RBrr |
|
— |
|
|
|
|
2 |
AMPSO2 |
|
|
— |
|
|
|
5 |
RBld |
|
|
|
— |
|
|
3 |
sPSO |
|
|
|
|
— |
|
6 |
BrBl |
|
|
|
|
|
— | 1 |
These finding were confirmed by the results of paired Wilcoxon test. The
The Wilcoxon test on best values achieved on
RBRa | RBrr | AMPSO2 | RBld | sPSO | BrBl | Ranking | |
---|---|---|---|---|---|---|---|
RBRa | — |
|
|
|
|
|
4 |
RBrr |
|
— |
|
|
|
|
2-3 |
AMPSO2 |
|
|
— |
|
|
|
5 |
RBld |
|
|
|
— |
|
|
2-3 |
sPSO |
|
|
|
|
— |
|
6 |
BrBl |
|
|
|
|
|
— | 1 |
The presented analysis evaluates the 6 different versions of PSO algorithm on 28 CEC2013 benchmark functions. The goal was to experimentally compare the different inertia weight updating strategies related to the random component generated by the Beta distribution.
The computational experiment consists of approximately
The results of comparison of selected single swarm PSO versions indicate that the Beta distribution applied on inertia weight strategy provides important source of modifications of original PSO. It supports the balanced exploratory and exploitive search. The best single swarm strategies according to the results of contrast test based on unadjusted median are RBld and RBrr.
Our results highlight that the best version from 6 tested PSO modifications is the multiswarm algorithm BrBl. The BrBl combines the swarms with modifications of inertia weight by the random component controlled by the time varied constant shape parameters and randomly varied shape parameters of Beta distributions.
The authors declare that there is no conflict of interests regarding the publication of this paper.