With a hypothesis that the social hierarchy of the grey wolves would be also followed in their searching positions, an improved grey wolf optimization (GWO) algorithm with variable weights (VW-GWO) is proposed. And to reduce the probability of being trapped in local optima, a new governing equation of the controlling parameter is also proposed. Simulation experiments are carried out, and comparisons are made. Results show that the proposed VW-GWO algorithm works better than the standard GWO, the ant lion optimization (ALO), the particle swarm optimization (PSO) algorithm, and the bat algorithm (BA). The novel VW-GWO algorithm is also verified in high-dimensional problems.
A lot of problems with huge numbers of variables, massive complexity, or having no analytical solutions were met during the behavior of exploring, exploiting, and conquering nature by human beings. The optimization methods are proposed to solve them. But unfortunately, because of the no free lunch rule [
Traditionally, the optimization algorithms are divided into two parts: the deterministic algorithms and the stochastic algorithms [
The NIC algorithms are proposed with inspiration of the nature, and they have been proved to be efficient to solve the problems human meet [
Almost all of the metaheuristic algorithms and their improvements so far are inspired directly from the behaviors of the organisms such as searching, hunting [
The GWO algorithm considers the searching, hunting behavior, and the social hierarchy of the grey wolves. Due to less randomness and varying numbers of individuals assigned in global and local searching procedures, the GWO algorithm is easier to use and converges more rapidly. It has been proved to be more efficient than the PSO [
Section
According to Mirjalili et al. [
Mirjalili designed the optimization algorithm imitating the searching and hunting process of grey wolves. In the mathematical model, the fittest solution is called the alpha (
When a prey is found, the iteration begins (
The parameters
The controlling parameter
The controlling parameter
Pseudocode of the GWO algorithm.
Description | Pseudocode |
---|---|
Set up optimization | Dimension of the given problems |
Limitations of the given problems | |
Population size | |
Controlling parameter | |
Stop criterion (maximum iteration times or admissible errors) | |
|
|
Initialization | Positions of all of the grey wolves including |
|
|
Searching | While not the stop criterion, calculate the new fitness function |
Update the positions | |
Limit the scope of positions | |
Refresh |
|
Update the stop criterion | |
End |
We can see from the governing equation ( The searching and hunting process are always governed by the alpha, the beta plays a less important role, and the delta plays a much less role. All of the other grey wolves transfer his/her position to the alpha if he/she gets the best. It should be noted that, in real searching and hunting procedures, the best position is nearest to the prey, while in optimization for a global optimum of a given problem, the best position is the maximum or minimum of the fitness value under given restrictions. During the searching process, a hypothesized prey is always surrounded by the dominants, while in hunting process, a real prey is encircled. The dominant grey wolves are at positions surrounding the prey in order of their social hierarchy. This means that the alpha is the nearest one among the grey wolves; the beta is the nearest one in the pack except for the alpha; and the delta ranks the third. The omega wolves are involved in the processes, and they transfer their better positions to the dominants.
With hypothesis mentioned hereinbefore, the update method of the positions should not be considered the same in equation (
When the search begins, the alpha is the nearest, and the rest are all not important. So, his/her position should be contributed to the new searching individuals, while all of the others could be ignored. This means that the weight of the alpha should be near to 1.0 at the beginning, while the weights of the beta and delta could be near zero at this time. At the final state, the alpha, beta, and the delta wolves should encircle the prey, which means they have an equal weight, as mentioned in equation (
The above ideas could be formulated in mathematics. First of all, all of the weights should be varied and limited to 1.0 when they are summed up. Equation (
Secondly, the weight of the alpha
Thirdly, the weights should be varied with the cumulative iteration number or “it”. And we know that
Considering
The curve of the variable weights is drawn in Figure
The variable weights vs. iterations.
In equation (
Furthermore, the controlling parameter is a restriction parameter for A, who is responsible for the grey wolf to approach or run away from the dominants. In other words, the controlling parameter governs the grey wolves to search globally or locally in the optimizing process. The global search probability is expected to be larger when the search begins; and consequently, the local search probability is expected to be larger when the algorithm is approaching the optimum. Therefore, to obtain a better performance of the GWO algorithm, the controlling parameter is expected to be decreased quickly when the optimization starts and converge to the optimum very fast. On the contrary, some grey wolves are expected to remain global searching to avoid being trapped in local optima. Considering these reasons, a controlling parameter declined exponentially [
The goal of experiments is to verify the advantages of the improved GWO algorithm with variable weights (VW-GWO) with comparisons to the standard GWO algorithm and other metaheuristic algorithms in this paper. Classically, optimization algorithms are applied to optimize benchmark functions which were used to describe the real problems human meet.
Although there are less numbers of parameters in the GWO algorithm than that in other algorithms such as the ALO, PSO, and bat algorithm (BA) [
Relationship between the MLIT and maximum value
We can know from Figure
Benchmark functions are standard functions which are derived from the research on nature. They are usually diverse and unbiased, difficult to be solved with analytical expressions. The benchmark functions have been an essential way to test the reliability, efficiency, and validation of optimization algorithms. They varied from the number of ambiguous peaks in the function landscape, the shape of the basins or valleys, reparability to the dimensional. Mathematically speaking, the benchmark functions can be classified with the following five attributes [ Continuous or uncontinuous: most of the functions are continuous, but some of them are not. Differentiable or nondifferentiable: some of the functions can be differenced, but some of them not. Separable or nonseparable: some of the functions can be separated, but some of them are not. Scalable or nonscalable: some of the functions can be expanded to any dimensional, but some of them are fixed to two or three dimensionalities. Unimodal or multimodal: some of the functions have only one peak in their landscape, but some of them have many peaks. The former attribute is called unimodal, and the latter is multimodal.
There are 175 benchmark functions, being summarized in literature [
Benchmark functions to be fitted.
Label | Function name | Expressions | Domain [ |
---|---|---|---|
F1 | De Jong’s sphere |
|
[−100, 100] |
F2 | Schwefel’s problems 2.22 |
|
[−100, 100] |
F3 | Schwefel’s problem 1.2 |
|
[−100, 100] |
F4 | Schwefel’s problem 2.21 |
|
[−100, 100] |
F5 | Chung Reynolds function |
|
[−100, 100] |
F6 | Schwefel’s problem 2.20 |
|
[−100, 100] |
F7 | Csendes function |
|
[−1, 1] |
F8 | Exponential function |
|
[−1, 1] |
F9 | Griewank’s function |
|
[−100, 100] |
F10 | Salomon function |
|
[−100, 100] |
F11 | Zakharov function |
|
[−5, 10] |
The functions are all
There are 11 benchmark functions being involved in this study. Comparisons are made with the standard grey wolf optimization algorithm (std. GWO) and three other bionic methods such as the ant lion optimization algorithm (ALO), the PSO algorithm, and BA.
The randomness is all involved in the algorithms studied in this paper, for example, the random positions, random velocities, and random controlling parameters. The randomness causes the fitness values obtained during the optimization procedure to fluctuate. So, when an individual of the swarm is initialized or it randomly jumps to a position quite near the optimum, the best fitness value would be met. Table
The best and worst simulation results and their corresponding algorithms (dim = 2).
Functions | Value | Corresponding algorithm |
---|---|---|
|
||
F1 | 1.4238 |
VM-GWO |
F2 | 3.2617 |
VM-GWO |
F3 | 3.6792 |
VM-GWO |
F4 | 3.3655 |
Std. GWO |
F7 | 7.8721 |
VM-GWO |
F8 | 0 | VM-GWO, Std. GWO, PSO, BA |
F9 | 0 | VM-GWO, Std. GWO |
F11 | 2.6230 |
VM-GWO |
|
||
|
||
F1 | 1.0213 |
BA |
F2 | 4.1489 |
BA |
F3 | 5.9510 |
BA |
F4 | 2.4192 |
PSO |
F7 | 1.0627 |
BA |
F8 | 5.7010 |
BA |
F9 | 1.0850 |
ALO |
F11 | 9.9157 |
BA |
The GWO algorithms always work the best at first glance of Table
F3: convergence vs. iterations (dim = 2).
The convergence rate curve during the iterations of F3 benchmark function is demonstrated in Figure
General acquaintances of the metaheuristic algorithms might be got from Table
Statistical analysis on the absolute errors of the selected functions (dim = 2).
Functions | VM-GWO | Std. GWO | ALO | PSO | BA | |||||
---|---|---|---|---|---|---|---|---|---|---|
Mean | Std. deviation | Mean | Std. deviation | Mean | Std. deviation | Mean | Std. deviation | Mean | Std. deviation | |
F1 | 7.2039 |
3.5263 |
6.59 |
6.34 |
2.59 |
1.65 |
1.36 |
2.02 |
0.773622 | 0.528134 [ |
F2 | 1.3252 |
3.5002 |
7.18 |
0.02901 [ |
1.84241 |
6.58 |
0.042144 | 0.04542 [ |
0.334583 | 3.186022 [ |
F3 | 3.7918 |
1.1757 |
3.29 |
79.1496 [ |
6.0685 |
6.34 |
70.12562 | 22.1192 [ |
0.115303 | 0.766036 [ |
F4 | 2.2262 |
2.8758 |
5.61 |
1.31509 [ |
1.36061 |
1.81 |
0.31704 | 7.3549 [ |
0.192185 | 0.890266 [ |
F5 | 3.6015 |
9.0004 |
7.8319 |
2.4767 |
2.1459 |
2.8034 |
8.4327 |
1.7396 |
1.7314 |
4.9414 |
F9 | 0.0047 | 0.0040 | 0.00449 | 0.00666 [ |
0.0301 | 0.0329 | 0.00922 | 0.00772 [ |
0.0436 | 0.0294 |
F10 | 0.0200 | 0.0421 | 0.0499 | 0.0526 | 0.01860449 | 0.009545 [ |
0.273674 | 0.204348 [ |
1.451575 | 0.570309 [ |
F11 | 1.2999 |
4.1057 |
6.8181 |
1.5724 |
1.1562 |
1.2486 |
2.3956 |
3.6568 |
5.0662 |
4.9926 |
The proposed VM-GWO algorithm and its compared algorithms are almost all capable of searching the global optima of the benchmark functions. The detailed values in Table All of the algorithms involved in this study were able to find the optimum. All of the benchmark functions tested in this experiment could be optimized, whether they are unimodal or multimodal, under the symmetric or unsymmetric domain. Comparatively speaking, although the bat algorithm is composed of much more randomness, it did the worst job. The PSO and the ALO algorithm did a little better. The GWO algorithms implement the optimization procedure much better. The proposed VM-GWO algorithm optimized most of the benchmark functions involved in this simulation at the best, and it did much better than the standard algorithm.
Therefore, the proposed VM-GWO algorithm is better performed in optimizing the benchmark functions than the std. GWO algorithm as well as the ALO, PSO algorithm, and the BA, which can be also obtained from the Wilcoxon rank sum test [
F1 | F2 | F3 | F4 | F5 | F6 | F7 | F8 | F9 | F10 | F11 | |
---|---|---|---|---|---|---|---|---|---|---|---|
Std. GWO | 0.000246 | 0.00033 | 0.000183 | 0.00044 | 0.000183 | 0 | 0.000183 | — | 0.466753 | 0.161972 | 0.000183 |
PSO | 0.000183 | 0.000183 | 0.000183 | 0.000183 | 0.472676 | 0 | 0.000183 | 0.167489 | 0.004435 | 0.025748 | 0.000183 |
ALO | 0.000183 | 0.000183 | 0.000183 | 0.000183 | 0.472676 | 0 | 0.000183 | 0.36812 | 0.790566 | 0.025748 | 0.000183 |
BA | 0.000183 | 0.000183 | 0.000183 | 0.000183 | 0.000183 | 0 | 0.000183 | 0.000747 | 0.004435 | 0.01133 | 0.000183 |
In Table
Compared with other bionic algorithms, the GWO algorithm has fewer numbers of parameter. Compared with the std. GWO algorithm, the proposed VM-GWO algorithm does not generate additional uncontrolling parameters. It furthermore improves the feasibility of the std. GWO algorithm by introducing an admissible maximum iteration number. On the contrary, there are large numbers of randomness in the compared bionic algorithms such as the ALO, PSO algorithms, and the BA. Therefore, the proposed algorithm is expected to be fond by the engineers, who need the fastest convergence, the most precise results, and which are under most control. Thus, there is a need to verify the proposed algorithm to be fast convergent, not only a brief acquaintance from Figure
Generally speaking, the optimization algorithms are usually used to find the optima under constrained conditions. The optimization procedure must be ended in reality, and it is expected to be as faster as capable. The admissible maximum iteration number
MLITs and statistical results for F1.
dim | Algorithm | Best | Worst | Mean |
|
Std. deviation | Number |
---|---|---|---|---|---|---|---|
2 | VW-GWO | 6 | 12 | 9.90 | 1.7180 |
1.0493 | 100 |
Std. GWO | 7 | 13 | 10.38 | 1.8380 |
1.2291 | 100 | |
PSO | 48 | 1093 | 357.97 | 3.2203 |
205.3043 | 100 | |
BA | 29 | 59 | 41.00 | 1.3405 |
5.8517 | 100 | |
|
|||||||
10 | VW-GWO | 53 | 66 | 59.97 | 4.1940 |
2.7614 | 100 |
Std. GWO | 74 | 89 | 80.40 | 1.9792 |
2.7614 | 100 | |
PSO | 5713 | 11510 | 9279.22 | 2.9716 |
1300.8485 | 88 | |
BA | 6919 | 97794 | 44999.04 | 7.5232 |
25133.3096 | 78 | |
|
|||||||
30 | VW-GWO | 55 | 67 | 59.85 | 1.2568 |
2.4345 | 100 |
Std. GWO | 71 | 86 | 80.07 | 2.6197 |
3.3492 | 100 | |
PSO | 5549 | 12262 | 9314.78 | 9.6390 |
1316.3384 | 96 | |
BA | 7238 | 92997 | 44189.16 | 5.2685 |
24831.7443 | 79 |
MLITs and statistical results for F7.
dim | Algorithm | Best | Worst | Mean |
|
Std. deviation | Number |
---|---|---|---|---|---|---|---|
2 | VW-GWO | 1 | 3 | 1.46 | 6.3755 |
0.5397 | 100 |
Std. GWO | 1 | 2 | 1.41 | 1.0070 |
0.4943 | 100 | |
PSO | 2 | 2 | 2.00 | 0 | 0 | 100 | |
BA | 1 | 3 | 1.02 | 8.5046 |
0.200 | 100 | |
|
|||||||
10 | VW-GWO | 5 | 9 | 7.65 | 5.7134 |
0.9468 | 100 |
Std. GWO | 5 | 11 | 7.48 | 5.1288 |
1.1413 | 100 | |
PSO | 4 | 65 | 24.23 | 1.6196 |
10.9829 | 100 | |
BA | 13 | 49 | 25.29 | 5.9676 |
6.2366 | 100 | |
|
|||||||
30 | VW-GWO | 13 | 22 | 17.14 | 9.6509 |
1.7980 | 100 |
Std. GWO | 15 | 30 | 20.80 | 1.3043 |
2.6208 | 100 | |
PSO | 54 | 255 | 133.32 | 5.7600 |
42.5972 | 100 | |
BA | 40 | 101 | 62.68 | 1.8501 |
11.8286 | 100 |
MLITs and statistical results for F11.
dim | Algorithm | Best | Worst | Mean |
|
Std. deviation | Number |
---|---|---|---|---|---|---|---|
2 | VW-GWO | 3 | 9 | 6.63 | 5.6526 |
1.2363 | 100 |
Std. GWO | 4 | 10 | 6.66 | 3.5865 |
1.2888 | 100 | |
PSO | 6 | 125 | 46.35 | 1.6006 |
26.0835 | 100 | |
BA | 5 | 62 | 27.58 | 1.6166 |
11.0080 | 100 | |
|
|||||||
10 | VW-GWO | 10 | 200 | 65.57 | 2.8562 |
43.2281 | 100 |
Std. GWO | 14 | 246 | 68.68 | 2.6622 |
41.7104 | 100 | |
PSO | 15 | 1356 | 231.74 | 1.2116 |
257.1490 | 94 | |
BA | 15 | 214 | 113.19 | 5.1511 |
66.9189 | 100 | |
|
|||||||
30 | VW-GWO | 49 | 1179 | 312.24 | 1.2262 |
194.7643 | 100 |
Std. GWO | 65 | 945 | 294.45 | 3.1486 |
160.7119 | 100 | |
PSO | 32 | 5005 | 1086.11 | 6.0513 |
980.3386 | 72 | |
BA | 66 | 403 | 221.60 | 1.9072 |
40.5854 | 100 |
Table
Note that, in this experiment, the dimensions of the benchmark functions are varied from 2 to 10 and 30. The final results also show that if the dimensions of the benchmark functions are raised, the MLIT values would be increased dramatically. This phenomenon would lead to the doubt whether it also performs the best and is capable to solve high-dimensional problems.
Tables
Statistical analysis on the absolute errors of the selected functions (dim = 200).
Functions | VM-GWO | Std. GWO | ALO | PSO | BA | |||||
---|---|---|---|---|---|---|---|---|---|---|
Mean | Std. deviation | Mean | Std. deviation | Mean | Std. deviation | Mean | Std. deviation | Mean | Std. deviation | |
F4 | 3.3556 |
8.7424 |
1.6051 |
2.2035 |
4.2333 |
2.9234 |
3.0178 |
6.5449 |
1.6401 |
2.1450 |
F8 | 0 | 0 | 0 | 0 | 3.3307 |
7.4934 |
1.1102 |
3.5108 |
1.4466 |
1.9684 |
F11 | 0.0115 | 0.0193 | 0.0364 | 0.0640 | 8.3831 | 10.3213 | 12.6649 | 13.0098 | 4.7528 |
2.8097 |
The data listed in Table
To test its capability even further, we also carry out an experiment to verify the capability solving some benchmark function in high dimensions with restrictions MC = 100 and MLIT = 500. In this experiment, we change the dimensions from 100 to 1000, and the final results which are also the absolute errors averaged over MC times, being shown in Figure
Absolute errors vs
We can see from Figure
In this paper, an improved grey wolf optimization (GWO) algorithm with variable weights (VW-GWO algorithm) is proposed. A hypothesize is made that the social hierarchy of the packs would also be functional in their searching positions. And variable weights are then introduced to their searching process. To reduce the probability of being trapped in local optima, a governing equation of the controlling parameter is introduced, and thus, it is declined exponentially from the maximum. Finally, three types of experiments are carried out to verify the merits of the proposed VW-GWO algorithm. Comparisons are made to the original GWO and the ALO, PSO algorithm, and BA.
All the selected experiment results show that the proposed VW-GWO algorithm works better under different conditions than the others. The variance of dimensions cannot change its first position among them, and the proposed VW-GWO algorithm is expected to be a good choice to solve the large-scale problems.
However, the proposed improvements are mainly focusing on the ability to converge. It leads to faster convergence and wide applications. But it is not found to be capable for all the benchmark functions. Further work would be needed to tell the reasons mathematically. Other initializing algorithms might be needed to let the initial swarm individuals spread all through the domain, and new searching rules when the individuals are at the basins would be another hot spot of future work.
The simulation platform, as described in Section 3.3, is run on an assembled desktop computer being configured as follows: CPU: Xeon E3-1231 v3; GPU: NVidia GeForce GTX 750 Ti; memory: DDR3 1866 MHz; motherboard: Asus B85-Plus R2.0; hard disk: Kingston SSD.
The associate software of this paper could be downloaded from
The authors declare that they have no conflicts of interest.
Zheng-Ming Gao formulated the governing equations of variable weights, constructed the work, and wrote the paper. Juan Zhao proposed the idea on the GWO algorithm and programmed the work with Matlab. Her major contribution is in the programmed work and the proposed declined exponentially governing equations of the controlling parameter. Juan Zhao contributed equally to this work.
This work was supported in part by Natural Science Foundation of Jingchu University of Technology with grant no. ZR201514 and the research project of Hubei Provincial Department of Education with grant no. B2018241.