Stochastic global optimization (SGO) algorithms such as the particle swarm optimization (PSO) approach have become popular for solving unconstrained global optimization (UGO) problems. The PSO approach, which belongs to the swarm intelligence domain, does not require gradient information, enabling it to overcome this limitation of traditional nonlinear programming methods. Unfortunately, PSO algorithm implementation and performance depend on several parameters, such as cognitive parameter, social parameter, and constriction coefficient. These parameters are tuned by using trial and error. To reduce the parametrization of a PSO method, this work presents two efficient hybrid SGO approaches, namely, a real-coded genetic algorithm-based PSO (RGA-PSO) method and an artificial immune algorithm-based PSO (AIA-PSO) method. The specific parameters of the internal PSO algorithm are optimized using the external RGA and AIA approaches, and then the internal PSO algorithm is applied to solve UGO problems. The performances of the proposed RGA-PSO and AIA-PSO algorithms are then evaluated using a set of benchmark UGO problems. Numerical results indicate that, besides their ability to converge to a global minimum for each test UGO problem, the proposed RGA-PSO and AIA-PSO algorithms outperform many hybrid SGO algorithms. Thus, the RGA-PSO and AIA-PSO approaches can be considered alternative SGO approaches for solving standard-dimensional UGO problems.
An unconstrained global optimization (UGO) problem can generally be formulated as follows:
Many conventional nonlinear programming (NLP) techniques, such as the golden search, quadratic approximation, Nelder-Mead, steepest descent, Newton, and conjugate gradient methods, have been used to solve UGO problems [
This work focuses on a PSO algorithm, based on it is being effective, robust and easy to use in the SGO methods. Research on the PSO method has considered many critical issues such as parameter selection, integration of the PSO algorithm with the approaches of self-adaptation, and integration with other intelligent optimizing methods [
Regarding the first issue, the conventional PSO algorithm lacks evolution operators of GAs, such as crossover and mutation operations. Therefore, PSO has premature convergence, that is, a rapid loss of diversity during optimization [
Regarding the second issue, a PSO algorithm has numerous parameters that must be set, such as cognitive parameter, social parameter, inertia weight, and constriction coefficient. Traditionally, the optimal parameter settings of a PSO algorithm are tuned based on trial and error. The abilities of a PSO algorithm to explore and exploit are constrained to optimum parameter settings [
This work focuses on the second issue related to the application of a PSO method. Fortunately, the optimization of parameter settings for a PSO algorithm can be viewed as an UGO problem. Moreover, real-coded GA (RGA) and AIA are efficient SGO approaches for solving UGO problems. Based on the advantage of a hybrid algorithm [
The rest of this paper is organized as follows. Section
The SGO approaches such as RGA, PSO, and AIA [
GAs are based on the concepts of natural selection and use three genetic operations, that is, selection, crossover, and mutation, to explore and exploit the solution space. In solving continuous function optimization problems, RGA method outperforms binary-coded GA approach [
A selection operation picks up strong individuals from a current population based on their fitness function values and then reproduces these individuals into a crossover pool. Many selection operations developed include the roulette wheel, the ranking, and the tournament methods [
While exploring the solution space by creating new offspring, the crossover operation randomly chooses two parents from the crossover pool and then uses these two parents to create two new offspring. This operation is repeated until the psRGA/2 is satisfied. The whole arithmetic crossover is easily performed as follows:
Mutation operation can improve the diversity of individuals (candidate solutions). Multi-non-uniform mutation is described as follows:
Kennedy and Eberhart [
The particle positions can be obtained using (
Shi and Eberhart [
A constriction coefficient (
This work considers parameters
According to (
Wu [
In the human system, an antigen (
The selection operation controls the number of antigen-specific
The
The somatic hypermutation operation can be expressed as follows:
This operation has two tasks, that is, a uniform search and local fine tuning.
A receptor-editing operation is developed based on the standard Cauchy distribution
This operation is used in local fine-tuning and large perturbation.
The paratope of an
This work develops the RGA-PSO and AIA-PSO approaches for solving UGO problems. The implementation of the RGA-PSO and AIA-PSO methods is described as follows.
Figure
The pseudocode of the proposed RGA-PSO algorithm.
Candidate solution of the RGA method.
The candidate solution
Internal steps from
External steps 2 to 6 are repeated until the
Figure
The pseudocode of the proposed AIA-PSO algorithm.
Internal steps from
Consistent with the
Following the evaluation of the
Repeat external Steps 2–6 until the termination criterion
The proposed RGA-PSO and AIA-PSO algorithms were applied to a set of benchmark UGO problems taken from other studies [
The parameter settings for the proposed RGA-PSO and AIA-PSO approaches.
Methods | Parameter settings | Search space |
---|---|---|
External RGA |
|
|
External AIA |
|
|
| ||
Internal PSO | psPSO = 20 |
|
Table
Numerical results obtained from the proposed RGA-PSO algorithm for solving 14 UGO problems.
TP number | Function name | Global minimum | Required accuracy | Success rate (%) |
|
|
|
|
MCCT (sec.) |
---|---|---|---|---|---|---|---|---|---|
1 | SHCB | −1.0316 |
|
100 | −1.0316 | −1.0316 | −1.0316 | 0.00000000 | 40.56 |
2 | GP | 3 |
|
100 | 3 | 3 | 3 | 0.00000000 | 38.29 |
3 | ES | −1 |
|
100 | −1 | −1 | −1 | 0.00000000 | 41.29 |
4 | B2 | 0 |
|
100 | 0.00000000 | 0.00000000 | 0.00000000 | 0.00000000 | 40.26 |
5 | DJ | 0 |
|
100 | 0.00000000 |
|
|
|
40.50 |
6 | Booth | 0 |
|
100 | 0.00000000 | 0.00000000 | 0.00000000 | 0.00000000 | 37.57 |
7 | RC | 5/ |
|
100 | 0.39788735 | 0.39788735 | 0.39788735 | 0.00000000 | 40.97 |
8 | RA | −2 |
|
100 | −2 | −2 | −2 | 0.00000000 | 39.31 |
9 | RS2 | 0 |
|
100 | 0.00000000 |
|
|
|
37.47 |
10 | RS5 | 0 |
|
100 |
|
|
|
|
306.15 |
11 | SH | −186.7309 |
|
100 | −186.7309 | −186.7309 | −186.7309 | 0.00000000 | 54.11 |
12 | ZA2 | 0 |
|
100 | 0.00000000 | 0.00000000 | 0.00000000 | 0.00000000 | 40.74 |
13 | ZA5 | 0 |
|
100 | 0.00000000 |
|
|
|
157.09 |
14 | ZA10 | 0 |
|
100 |
|
|
|
|
194.73 |
The optimal parameter settings obtained using the proposed RGA-PSO algorithm for solving 14 UGO problems.
TP number | Function name |
|
|
|
---|---|---|---|---|
1 | SHCB | 0.36497798 | 1.79972890 | 2.36717639 |
2 | GP | 0.86138264 | 3.74964714 | 0.10000000 |
3 | ES | 0.36038653 | 0.42043804 | 4.78367612 |
4 | B2 | 0.75486133 | 0.17875492 | 0.95144705 |
5 | DJ | 0.39357143 | 3.17898242 | 4.72512283 |
6 | Booth | 0.56703400 | 4.84740898 | 2.58109482 |
7 | RC | 0.44153938 | 2.71748076 | 3.81076900 |
8 | RA | 0.93533829 | 0.76530619 | 3.53402840 |
9 | RS2 | 0.87003776 | 1.28577955 | 2.77423057 |
10 | RS5 | 0.62553666 | 0.10000000 | 5.00000000 |
11 | SH | 0.45903159 | 1.07452631 | 4.15038556 |
12 | ZA2 | 0.75044069 | 0.50883133 | 1.93860581 |
13 | ZA5 | 0.52013863 | 1.05471011 | 4.61197390 |
14 | ZA10 | 0.56126516 | 0.39456605 | 4.95735921 |
Table
Numerical results obtained from the proposed AIA-PSO algorithm for solving 14 UGO problems.
TP number | Function name | Global minimum | Required accuracy | Success rate (%) |
|
|
|
|
MCCT (sec) |
---|---|---|---|---|---|---|---|---|---|
1 | SHCB | −1.0316 |
|
100 | −1.0316 | −1.0316 | −1.0316 | 0.00000000 | 39.69 |
2 | GP | 3 |
|
100 | 3 | 3 | 3 | 0.00000000 | 37.80 |
3 | ES | −1 |
|
100 | −1 | −1 | −1 | 0.00000000 | 42.08 |
4 | B2 | 0 |
|
100 | 0.00000000 | 0.00000000 | 0.00000000 | 0.00000000 | 40.43 |
5 | DJ | 0 |
|
100 | 0.00000000 |
|
|
|
39.43 |
6 | Booth | 0 |
|
100 | 0.00000000 | 0.00000000 | 0.00000000 | 0.00000000 | 37.48 |
7 | RC | 5/ |
|
100 | 0.39788735 | 0.39788735 | 0.39788735 | 0.00000000 | 40.53 |
8 | RA | −2 |
|
100 | −2 | −2 | −2 | 0.00000000 | 39.17 |
9 | RS2 | 0 |
|
100 | 0.00000000 |
|
|
|
37.49 |
10 | RS5 | 0 |
|
100 |
|
|
|
|
304.23 |
11 | SH | −186.7309 |
|
100 | −186.7309 | −186.7309 | −186.7309 | 0.00000000 | 54.70 |
12 | ZA2 | 0 |
|
100 | 0.00000000 | 0.00000000 | 0.00000000 | 0.00000000 | 40.31 |
13 | ZA5 | 0 |
|
100 | 0.00000000 |
|
|
|
144.38 |
14 | ZA10 | 0 |
|
100 |
|
|
|
|
192.06 |
The optimal parameter settings obtained using the proposed AIA-PSO algorithm for solving 14 UGO problems.
TP number | Function name |
|
|
|
---|---|---|---|---|
1 | SHCB | 0.59403239 | 1.75960560 | 4.12826475 |
2 | GP | 0.29584147 | 2.62468918 | 4.41787729 |
3 | ES | 0.39500323 | 0.84589484 | 2.64979988 |
4 | B2 | 0.64806710 | 1.78727677 | 1.83995777 |
5 | DJ | 0.45630341 | 1.11829588 | 4.10382418 |
6 | Booth | 0.60889449 | 2.53158038 | 1.32827344 |
7 | RC | 1.00000000 | 1.48772085 | 0.43921335 |
8 | RA | 0.38157302 | 4.93248942 | 4.08732827 |
9 | RS2 | 0.39872504 | 4.65992304 | 4.84228028 |
10 | RS5 | 0.55519288 | 0.10000000 | 4.52483714 |
11 | SH | 0.54293231 | 2.10545153 | 4.52889414 |
12 | ZA2 | 0.44530811 | 1.05474921 | 3.20092987 |
13 | ZA5 | 0.51683760 | 1.17129205 | 4.94965004 |
14 | ZA10 | 0.53688659 | 0.53752883 | 4.99881022 |
To investigate the effectiveness of the RGA-PSO and AIA-PSO methods for solving a standard-dimensional UGO problem, the Zakharov problem with 30 decision variables (ZA30), as described in the Appendix, has been solved using the RGA-PSO and AIA-PSO approaches. Fifty independent runs were performed to solve the UGO problem. To increase the diversity of candidate solutions for use in the external RGA method, the parameter
Numerical results obtained from the proposed RGA-PSO and AIA-PSO algorithms for solving a standard-dimensional UGO problem.
Function |
Global minimum | Required accuracy | Methods | Success rate (%) |
|
|
|
|
MCCT (sec) |
---|---|---|---|---|---|---|---|---|---|
ZA30 | 0 |
|
RGA-PSO | 100 |
|
|
|
|
1753.86 |
AIA-PSO | 100 |
|
|
|
|
1453.98 |
The optimal parameter settings obtained using the proposed RGA-PSO and AIA-PSO algorithms for solving a standard-dimensional UGO problem.
Function name | Methods |
|
|
|
---|---|---|---|---|
ZA30 | RGA-PSO | 0.58721675 | 0.56183025 | 4.91401312 |
AIA-PSO | 0.61082112 | 0.38556559 | 5.00000000 |
The UGO problem ZA50 was solved using the RGA-PSO and AIA-PSO methods. The RGA-PSO and AIA-PSO methods fail to solve the UGO problem, since the diversity of the particle swarm in the internal PSO method cannot be maintained. Hence, future work will focus on improving the diversity of the particle swarm by applying mutation operations.
Table
Results of Wilcoxon test for the MEs obtained using the proposed RGA-PSO and AIA-PSO methods for 14 UGO problems.
TP number | Function name | RGA-PSO versus AIA-PSO |
---|---|---|
| ||
1 | SHCB | ** |
2 | GP | ** |
3 | ES | ** |
4 | B2 | ** |
5 | DJ | ** |
6 | Booth | ** |
7 | RC | ** |
8 | RA | ** |
9 | RS2 | ** |
10 | RS5 | 0.833 |
11 | SH | ** |
12 | ZA2 | ** |
13 | ZA5 | ** |
14 | ZA10 | ** |
Table
Comparison of the results of the proposed RGA-PSO and AIA-PSO approaches and those of the hybrid algorithms for 11 TPs.
TP number | Function |
| |||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
NM-PSO [ |
PSACO [ |
GA-PSO [ |
DE-PSO [ |
AM-PSO1 [ |
AM-PSO2 [ |
CHA [ |
CTSS [ |
AIA-PSO |
RGA-PSO | ||
2 | GP | 0.00003 | 0.00000000 | 0.00012 |
|
|
|
0.0010 | 0.001 |
|
|
3 | ES | 0.00004 | 0.00000000 | 0.00003 |
|
|
|
0.0010 | 0.005 |
|
|
4 | B2 | 0.00003 |
|
0.00001 |
|
|
|
0.0000002 | 0.000005 |
|
|
5 | DJ | 0.00003 |
|
0.00004 |
|
|
|
0.0002 | 0.0002 |
|
|
7 | RC | 0.00003 |
|
0.00009 |
|
|
|
0.0001 | 0.005 |
|
|
9 | RS2 | 0.00003 |
|
0.00064 |
|
|
|
0.0040 | 0.0040 |
|
|
10 | RS5 | 0.0056 |
|
0.00013 |
|
|
|
— | — |
|
|
11 | SH | 0.00002 |
|
0.00007 |
|
|
|
0.0050 | 0.001 |
|
|
12 | ZA2 | 0.00003 |
|
0.00005 |
|
|
|
|
|
|
|
13 | ZA5 | 0.00026 |
|
0.00000 |
|
|
|
— | — |
|
|
14 | ZA10 | — |
|
0.00000 |
|
|
|
|
— |
|
|
(“—” denotes unavailable information).
Comparison of the success rate % of the proposed RGA-PSO and AIA-PSO approaches and those of the hybrid algorithms for 11 TPs.
TP number | Function name | Success rate % | |||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
NM-PSO [ |
PSACO [ |
GA-PSO [ |
DE-PSO [ |
AM-PSO1 [ |
AM-PSO2 [ |
CHA [ |
CTSS [ |
AIA-PSO |
RGA-PSO | ||
2 | GP | 100 | 100 | 100 | 100 | 100 | 100 | 100 | 100 | 100 | 100 |
3 | ES | 100 | 100 | 100 | 100 | 100 | 100 | 100 | 100 | 100 | 100 |
4 | B2 | 100 | 100 | 100 | 100 | 100 | 100 | 100 | 100 | 100 | 100 |
5 | DJ | 100 | 100 | 100 | 100 | 100 | 100 | 100 | 100 | 100 | 100 |
7 | RC | 100 | 100 | 100 | 100 | 100 | 100 | 100 | 100 | 100 | 100 |
9 | RS2 | 100 | 100 | 100 | 100 | 100 | 100 | 100 | 100 | 100 | 100 |
10 | RS5 | 100 | 100 | 100 | 100 | 100 | 100 | — | — | 100 | 100 |
11 | SH | 100 | 100 | 100 | 100 | 100 | 100 | 100 | 100 | 100 | 100 |
12 | ZA2 | 100 | 100 | 100 | 100 | 100 | 100 | 100 | 100 | 100 | 100 |
13 | ZA5 | 100 | 100 | 100 | 100 | 100 | 100 | — | — | 100 | 100 |
14 | ZA10 | 100 | 100 | 100 | 100 | 100 | 100 | 100 | — | 100 | 100 |
(“—” denotes unavailable information).
The proposed RGA-PSO and AIA-PSO algorithms have the following benefits. Parameter manipulation of the internal PSO algorithm is based on the solved UGO problems. Owing to their ability to efficiently solve UGO problems, the external RGA and AIA approaches are substituted for trial and error to manipulate the parameters ( Besides obtaining the optimum parameter settings of the internal PSO algorithm, the RGA-PSO and AIA-PSO algorithms can yield a global minimum for an UGO problem. Beside, outperforming some published hybrid SGO methods, the proposed RGA-PSO and AIA-PSO approaches reduce the parametrization for the internal PSO algorithm, despite being more complex than individual SGO approaches.
The proposed RGA-PSO and AIA-PSO algorithms are limited in that they cannot solve high-dimensional UGO problems (such as
This work developed RGA-PSO and AIA-PSO algorithms. Performances of the proposed RGA-PSO and AIA-PSO approaches were evaluated using a set of benchmark UGO problems. Numerical results indicate that the proposed RGA-PSO and AIA-PSO methods can converge to global minimum for each test UGO problem and obtain the best parameter settings of the internal PSO algorithm. Moreover, the numerical results obtained using the RGA-PSO and AIA-PSO algorithms are superior to those obtained using many alternative hybrid SGO methods. The RGA-PSO and AIA-PSO methods can thus be considered efficient SGO approaches for solving standard-dimensional UGO problems.
(1) search domain: one global minimum at two different points:
(2) search domain: four local minima; one global minimum:
(3) search domain: several local minima (exact number unspecified in usual literature); one global minimum:
(4) search domain: several local minima (exact number unspecified in usual literature); one global minimum
(5) search domain: one global minimum:
(6) search domain: one global minimum:
(7) search domain: no local minimum; three global minima:
(8) search domain: 50 local minima; One global minimum:
(9) Two functions were considered: RS2, RS5 search domain: several local minima (exact number unspecified in usual literature); global minimum:
(10) search domain: 760 local minima; 18 global minima;
(11) Three functions were considered: ZA2, ZA5, ZA10, ZA30 search domain: several local minima (exact number unspecified in usual literature); global minimum:
The author confirm that he does not has a conflict of interest with the MATLAB software.
The author would like to thank the National Science Council of the Republic of China, Taiwan for financially supporting this research under Contract no. NSC 100-2622-E-262-006-CC3.