Here, a two-phase algorithm is proposed for solving bounded continuous-time nonlinear optimal control problems (NOCP). In each phase of the algorithm, a modified hybrid genetic algorithm (MHGA) is applied, which performs a local search on offsprings. In first phase, a random initial population of control input values in time nodes is constructed. Next, MHGA starts with this population. After phase 1, to achieve more accurate solutions, the number of time nodes is increased. The values of the associated new control inputs are estimated by Linear interpolation (LI) or Spline interpolation (SI), using the curves obtained from the phase 1. In addition, to maintain the diversity in the population, some additional individuals are added randomly. Next, in the second phase, MHGA restarts with the new population constructed by above procedure and tries to improve the obtained solutions at the end of phase 1. We implement our proposed algorithm on 20 well-known benchmark and real world problems; then the results are compared with some recently proposed algorithms. Moreover, two statistical approaches are considered for the comparison of the LI and SI methods and investigation of sensitivity analysis for the MHGA parameters.
NOCPs are dynamic optimization problems with many applications in industrial processes such as airplane, robotic arm, bio-process system, biomedicine, electric power systems, and plasma physics [
High-quality solutions and the less required computational time are main issues for solving NOCPs. The numerical methods, direct [
Metaheuristics as the global optimization methods can overcome these problems, but they usually need more computational time, though they do not really need good initial guesses and deterministic rules. Several researchers used metaheuristics to solve optimal control problems. For instance, Michalewicz et al. [
To increase the quality of solutions and decrease the running time, hybrid methods were introduced, which used a local search in the implementation of a population-based metaheuristics, [
Based on the success of the hybrid methods for solving NOCPs, mentioned above, we here use a modified hybrid genetic algorithm (MHGA), which combines GA with SQP, see [
To perform MHGA for solving an NOCP, the time interval is uniformly divided by using a constant number of time nodes. Next, in each of these time nodes, the control variable is approximated by a scaler vector of control input values. Thus, an infinite dimensional NOCP is changed to a finite dimensional NLP. Now, we encounter two conflict situations: the quality of the global solution and the needed computational time. In other words, when the number of time nodes is increased, then we expect that the quality of the global solution is also increased, but we know that in this situation the computational time is increased dramatically. In other situation, if we consider less number of time nodes, then the computational time is decreased but we may find a poor local solution. To conquer these problems, MHGA performs in two phases. In the first phase (exploration phase), to decrease the computational time and to find a promising region of search space, MHGA uses a less number of time nodes. After phase 1, to increase the quality of solutions obtained from phase 1, the number of time nodes is increased. Using the population obtained in phase 1, the values of the new control inputs are estimated by Linear or Spline interpolations. Next, in the second phase (exploitation phase), MHGA uses the solutions constructed by the above procedure, as an initial population.
The paper is organized as follows: in Section
The bounded continuous-time NOCP is considered as finding the control input
In this section, first MHGA, as a subprocedure for the main algorithm, is introduced. To perform MHGA, the control variables are discretized. Next, NOCP is changed into a finite dimensional NLP; see [
GAs, introduced by Holland in 1975, are heuristics and probabilistic methods [
Let The corresponding control input value at each time node
{ individuals from population, randomly. Let
Let Let Perform Tournament algorithm with size Perform Tournament algorithm with size
Select the following numbers Let Let
The maximum number of generations, Over a specified number of generations,
The fitness value in (
SQP, is an effective and iterative algorithm for the numerical solution of the constrained NLP problem. This technique is based on finding a solution to the system of nonlinear equations that arise from the first-order necessary conditions for an extremum of the NLP problem. Using an initial solution of NLP, Given an initial solution Construct the QP subproblem ( Compute the new point as Let
Here, in MHGA, SQP is used as the local search, and we use the maximum number of iterations as the main criterion for stopping SQP. In other words, we terminate SQP when it converges either to local solution or the maximum number of SQP’s iterations is reached.
In MHGA, GA uses a local search method to improve solutions. Here, we use SQP as a local search. Using SQP as a local search in the hybrid metaheuristic is common, for example, see [
MHGA can be seen as a multi start local search where initial solutions are constructed by GA. From another perspective, MHGA can be seen as a GA that the quality of its population is intensified by SQP. In the beginning of MHGA, a less number of iterations for SQP was used. Then, when the promising regions of search space were found by GA operators, we increase the number of iterations of SQP gradually. Using this approach, we may decrease the needed running time (in [
Finally, we give our modified MHGA, to find the global solution, by the flowchart in Figure
Flowchart of the MHGA algorithm.
Here, we give a new algorithm, which is a direct approach, based on MHGA, for solving NOCPs. The proposed algorithm has two main phases. In the first phase, we perform MHGA with a completely random initial population constructed by (
After phase 1, to maintain the property of individuals in the last population of phase 1 and to increase the accurately of solutions, we add some additional time nodes. When the number of time nodes is increased, it is estimated that the quality of solution obtained by numerical methods (e.g., Runge-Kutta and Simpson) is increased. Thus, we increase time nodes from
{
{ { and estimate the corresponding control input values of the new time nodes in each individual obtained from phase 1, using either Linear or Spline interpolation. Create {
In this section, to investigate the efficiency of the proposed algorithm, 20 well-known and real world NOCPs, as benchmark problems, are considered which are presented in terms of (
The numerical behaviour of algorithms can be studied from two view of points, the relative error of the performance index and the status of the final state constraints. Let
To solve any NOCP described in the Appendix, we must know the algorithm’s parameters including: MHGA’s parameters; including
The best numerical results in 12 different runs of the SI and LI methods for NOCPs in the Appendix.
Problem | Algorithm |
|
|
|
|
Time |
---|---|---|---|---|---|---|
VDP | CGA | 1.7404 |
|
|
|
501.28 |
LI |
|
|
|
|
282.82 | |
SI |
|
|
|
|
239.14 | |
|
||||||
CRP | CGA |
|
0.2835 |
|
|
501.28 |
LI |
|
|
|
|
| |
SI |
|
|
|
|
| |
|
||||||
FFRP | CGA | 83.63 | 1.3256 |
|
|
1413 |
LI |
|
|
|
|
| |
SI | 35.99 |
|
|
|
1343.40 | |
|
||||||
MSNIC | IPSO | 0.1727 | 0.0165 | — | — | 126.26 |
LI |
|
|
— | — |
| |
SI |
|
|
— | — |
| |
|
||||||
CSTCR | IPSO | 0.1355 | 0.2098 | — | — |
|
LI |
|
|
— | — |
| |
SI |
|
|
— | — |
| |
|
||||||
No. 6 | Bézier |
|
|
— | — | NR |
LI |
|
|
— | — |
| |
SI |
|
|
— | — |
| |
|
||||||
Number 7 | HPM | 0.2353 | 0.1677 |
|
0.1677 | NR |
LI |
|
|
|
|
213.92 | |
SI |
|
|
|
|
206.96 | |
|
||||||
Number 8 | SQP |
|
|
— | — | NR |
SUMT |
|
|
— | — | NR | |
LI |
|
|
— | — | 72.82 | |
SI |
|
|
— | — |
| |
|
||||||
Number 9 | SQP | 1.7950 | 0.0873 | — | — | NR |
SUMT | 1.7980 | 0.0891 | — | — | NR | |
LI |
|
|
— | — | 639.10 | |
SI |
|
|
— | — |
| |
|
||||||
Number 10 | SQP | 0.2163 | 0.3964 | — | — | NR |
SUMT | 0.1703 | 0.0994 | — | — | NR | |
LI |
|
|
— | — |
| |
SI |
|
|
— | — | 676.62 | |
|
||||||
Number 11 | SQP | 3.25 | 0.1648 | 0 | 0.1648 | NR |
SUMT | 3.25 | 0.1648 | 0 | 0.1648 | NR | |
LI |
|
|
|
|
147.78 | |
SI |
|
|
|
|
| |
|
||||||
Number 12 | SQP |
|
|
0 |
|
NR |
SUMT |
|
|
0 |
|
NR | |
LI |
|
|
|
|
| |
SI |
|
|
|
|
545.48 | |
|
||||||
Number 13 | SQP |
|
0.1748 | 0 | 0.1748 | NR |
SUMT |
|
0.1678 | 0 | 0.1678 | NR | |
LI |
|
|
|
|
425.99 | |
SI |
|
|
|
|
| |
|
||||||
Number 14 | SQP | 3.7220 | 0.0961 | 0 | 0.0961 | NR |
SUMT | 3.7700 | 0.1103 | 0 | 0.1103 | NR | |
LI |
|
|
|
|
1041.24 | |
SI | 3.3965 |
|
|
|
| |
|
||||||
Number 15 | SQP |
|
4.3368 | 0 | 4.3368 | NR |
SUMT |
|
3.8135 | 0 | 3.8135 | NR | |
LI |
|
|
|
|
333.54 | |
SI |
|
|
|
|
| |
|
||||||
Number 16 | SQP | 2.2120 | 0.0753 | 0 | 0.0753 | NR |
SUMT | 2.2080 | 0.0734 | 0 | 0.0734 | NR | |
LI |
|
|
|
|
| |
SI |
|
|
|
|
704.23 | |
|
||||||
Number 17 | SQP |
|
|
0 |
|
NR |
SUMT |
|
|
0 |
|
NR | |
LI |
|
|
|
|
321.83 | |
SI |
|
|
|
|
| |
|
||||||
Number 18 | SQP | 0.0368 | 0.1288 | — | — | NR |
SUMT | 0.0386 | 0.1840 | — | — | NR | |
LI |
|
|
— | — |
| |
SI |
|
|
— | — |
| |
|
||||||
Number 19 | SQP | 0.3439 | 4.9293 | 0 | 4.9293 | NR |
SUMT | 0.3428 | 4.9103 | 0 | 4.9103 | NR | |
LI |
|
|
|
|
| |
SI |
|
|
|
|
| |
|
||||||
Number 20 | SQP | 77.52 | 1.2554 | 0 | 1.2554 | NR |
SUMT | 76.83 | 1.2353 | 0 | 1.2353 | NR | |
LI |
|
|
|
|
815.19 | |
SI |
|
|
|
|
|
The problem parameters for NOCPs in the Appendix.
Parameters | Problem number | |||||||||
---|---|---|---|---|---|---|---|---|---|---|
1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | |
|
|
|
|
|
0 |
|
0 |
|
|
|
|
2 | 2 | 10 | 20 | 5 | 2 | 1 | 2 | 1 | 20 |
|
|
|
70 |
|
— | 1 | — |
|
— |
|
|
|
|
|
— | — | — | — | — | — |
|
|
||||||||||
Parameters | Problem no. | |||||||||
11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | |
|
||||||||||
|
|
|
|
|
|
|
|
|
|
|
|
5 | 1 | 2 |
|
1 | 3 | 30 | 1 | 10 | 10 |
|
10 |
|
— |
|
|
|
— |
|
|
|
|
|
|
|
— |
|
|
|
— |
|
|
The reported numerical results of the proposed algorithm, for each NOCP, include the value of performance index,
The algorithm was implemented in Matlab R2011a environment on a Notebook with Windows 7 Ultimate, CPU 2.53 GHz and 4.00 GB RAM. Also, to implement SQP in our proposed algorithm, we used “fmincon” in Matlab when the “Algorithm” was set to “SQP”. Moreover, we use composite Simpson’s method [
We use the following abbreviations to show the used interpolation method in our proposed algorithm: LI: linear interpolation. SI: spline interpolation.
For comparing the numerical results of the proposed algorithm two subsections are considered, comparison with some metaheuristic algorithms, in Section
The numerical results for the NOCPs numbers
The first NOCP in the Appendix is Van Der Pol Problem, VDP, which has two state variables and one control variable. VDP problem has a final state constraint, which is
The second NOCP in the Appendix is Chemical Reactor Problem, CRP, which has two state variables and one control variable. The results of the proposed algorithm, with the MHGA’s parameters as
The third NOCP in the Appendix is Free Floating Robot Problem, FFRP, which has six state variables and four control variables. FFRP has been solved by CGA and the proposed algorithm, with the MHGA’s parameters as
For the forth NOCP in the Appendix, which is a Mathematical System with Nonlinear Inequality Constraint, NSNIC, the numerical results are compared with IPSO. MSNIC contains an inequality constraint,
Subplots (a) and (b) in Figure
Graphical results of MSNIC problem using SI method: (a) convergence rate of the performance index and (b) the inequality constraint,
The fifth NOCP in the Appendix is a model of a nonlinear Continuous Stirred-tank Chemical Reactor, CSTCR. It has two state variables
For NOCPs numbers
The NOCP number
Figure
Graphical results for NOCP number
For NOCP number
For the NOCPs numbers
Figures
Convergence rate for the performance index of SI method for NOCP number
(a) Convergence rate for the performance index and (b) the inequality constraint of SI method, for NOCP number
(a) Convergence rate of the performance index and (b) the inequality constraint for SI method, for NOCP number
(a) Convergence rates of the performance index and (b) the error of final state constraint for SI method, for NOCP number
(a) Convergence rates of the performance index and (b) the error of final state constraint for SI method, for NOCP number
(a) Convergence rates of the performance index and (b) the error of final state constraint for SI method, for NOCP number
(a) Convergence rates of the performance index and (b) the error of final state constraint for SI method, for NOCP number
(a) Convergence rates of the performance index and (b) the error of final state constraint for SI method, for NOCP number
(a) Convergence rates of the performance index and (b) the error of final state constraint for SI method, for NOCP number
(a) Convergence rates of the performance index and (b) the error of final state constraint for SI method, for NOCP number
Convergence rate for the performance index of SI method for NOCP numbers
(a) Convergence rates of the performance index and (b) the error of final state constraint for SI method, for NOCP number
(a) Convergence rates of the performance index and (b) the error of final state constraint for SI method, for NOCP number
Table
To compare with CGA, the mean of relative error of
From Table
Therefore, based on this numerical study, we can conclude that the proposed algorithm outperform well-known numerical method. Since, the algorithms were not implemented on the same PC, the computational times of them are not competitive. Therefore, we did not give the computational times in bold in Table
In this section, two statistical analysis, based on the one-way analysis of variance (ANOVA), used for investigating the sensitivity of MHGA parameters, and Mann-Whitney, applied comparing LI and SI methods, are done, by the statistical software IBM SPSS version 21.
In order to survey the sensitivity of the MHGA parameters from the proposed algorithm, the VDP problem is selected, for example. The influence of these parameters are investigated for this NOCP on the dependent outputs consist of the performance index,
At first, we selected at least four constant value for each of parameters and then in each pose, 35 different runs were made, independently. The statistical analysis is done based on ANOVA. The descriptive statistics, which contains the number of different runs
Descriptive statistics for the sensitivity analysis of
Output |
|
|
Mean | S.d | Min | Max |
---|---|---|---|---|---|---|
|
11 | 35 | 1.5333 | 0.0069 | 1.5286 | 1.5598 |
21 | 35 | 1.5328 | 0.0097 | 1.5285 | 1.5849 | |
31 | 35 | 1.5329 | 0.0076 | 1.5285 | 1.5699 | |
41 | 35 | 1.5455 | 0.0607 | 1.5285 | 1.8449 | |
Total |
|
|
|
|
|
|
|
||||||
|
11 | 35 |
|
|
|
|
21 | 35 |
|
|
|
|
|
31 | 35 |
|
|
|
|
|
41 | 35 |
|
|
|
|
|
Total |
|
|
|
|
|
|
|
||||||
Time | 11 | 35 | 89.7838 | 27.2081 | 45.6614 | 155.4394 |
21 | 35 | 109.8147 | 46.0264 | 60.4192 | 284.0310 | |
31 | 35 | 116.3591 | 30.0227 | 77.2673 | 179.5103 | |
41 | 35 | 144.8290 | 39.3091 | 81.0425 | 284.9514 | |
Total |
|
|
|
|
|
|
|
||||||
|
11 | 35 | 0.0031 | 0.0045 | 0 | 0.0204 |
21 | 35 | 0.0028 | 0.0064 | 0 | 0.0369 | |
31 | 35 | 0.0029 | 0.0050 | 0 | 0.0271 | |
41 | 35 | 0.0111 | 0.0397 | 0 | 0.2070 | |
Total |
|
|
|
|
|
|
|
||||||
|
11 | 35 |
|
|
|
|
21 | 35 |
|
|
|
| |
31 | 35 |
|
|
|
| |
41 | 35 |
|
|
|
| |
Total |
|
|
|
|
|
Table From Table From the second row of Table From the third row of Table From the forth row of Table From the fifth row of Table The sensitivity analysis is similar to
From above cases, it is obvious that all parameters can be effect on the required computational time, except
Summary statistical data of ANOVA test for the parameters,
Parameters |
|
|
|
Time |
| |
---|---|---|---|---|---|---|
Test statistic ( |
|
1.294 | 1.296 | 0.624 | 10.79 | 1.296 |
|
1105.72 | 3.93 | 0.868 | 59.39 | 3.93 | |
|
1.945 | 1.835 | 1.271 | 4.317 | 1.835 | |
|
2.740 | 2.478 | 1.011 | 23.02 | 2.478 | |
|
3.541 | 3.591 | 0.491 | 0.890 | 3.591 | |
|
0.495 | 0.309 | 0.301 | 5.849 | 1.046 | |
|
||||||
|
|
0.280 | 0.279 | 0.601 | 0 | 0.279 |
|
0 | 0.005 | 0.489 | 0 | 0.005 | |
|
0.125 | 0.143 | 0.286 | 0.006 | 0.143 | |
|
0.021 | 0.034 | 0.413 | 0 | 0.034 | |
|
0.009 | 0.008 | 0.743 | 0.472 | 0.008 | |
|
0.739 | 0.819 | 0.877 | 0 | 0.386 |
To compare the efficiency of the LI and SI methods, for NOCPs in the Appendix, we used the Mann-Whitney nonparametric statistical test [
The mean of numerical results of 15 different runs for NOCPs in the Appendix, using LI and SI methods.
Problem | LI | SI | ||||||
---|---|---|---|---|---|---|---|---|
|
|
|
Time |
|
|
|
Time | |
VDP | 1.5977 |
|
|
177.20 | 1.6002 |
|
|
157.57 |
CRP | 0.0126 |
|
|
197.14 | 0.0119 |
|
|
195.32 |
FFPP | 47.74 |
|
0.3277 | 1332.80 | 53.51 |
|
0.4868 | 1379.89 |
MSNIC | 0.1702 | — | — | 136.78 | 0.1703 | — | — | 140.92 |
CSTCR | 0.1120 | — | — | 173.85 | 0.1120 | — | — | 176.06 |
Number 6 |
|
— | — | 105.41 |
|
— | — | 103.72 |
Number 7 | 0.2015 |
|
|
208.07 | 0.2020 |
|
|
210.19 |
Number 8 |
|
— | — | 63.76 |
|
— | — | 58.37 |
Number 9 | 1.6512 | — | — | 592.39 | 1.6512 | — | — | 581.25 |
Number 10 | 0.1550 | — | — | 631.53 | 0.1550 | — | — | 655.97 |
Number 11 | 2.9701 |
|
|
228.60 | 2.9701 |
|
|
232.76 |
Number 12 |
|
|
|
547.18 |
|
|
|
527.65 |
Number 13 | 0.0147 |
|
|
577.97 |
|
|
|
660.98 |
Number 14 | 3.4761 |
|
|
1045.85 | 3.4290 |
|
|
1087.67 |
Number 15 |
|
|
|
363 |
|
|
|
365 |
Number 16 | 2.0571 |
|
|
543.91 | 2.0571 |
|
|
544.31 |
Number 17 |
|
|
|
227.44 |
|
|
|
232.54 |
Number 18 | 0.0326 | — | — | 659.57 | 0.0326 | — | — | 708.97 |
Number 19 | 0.11588 |
|
0.6828 | 1618.13 | 0.1091 |
|
0.8808 | 1599.90 |
Number 20 | 52.42 | 0.0176 | 0.5429 | 841.65 | 44.64 |
|
0.3765 | 868.91 |
Results of Mann-Whitney test on relative errors of the pair (LI, SI) for
Method | Mean rank | Sum of ranks | Test statistics | Value |
---|---|---|---|---|
LI | 20.50 | 410.0 | Mann-Whitney |
200 |
SI | 20.50 | 410.0 | Wilcoxon |
410 |
— | — | — |
|
0 |
Results of Mann-Whitney test on relative errors of the pair (LI, SI) for
Method | Mean rank | Sum of ranks | Test statistics | Value |
---|---|---|---|---|
LI | 13.62 | 177 | Mann-Whitney |
83 |
SI | 13.38 | 174 | Wilcoxon |
174 |
— | — | — |
|
|
Results of Mann-Whitney test on relative errors of the pair (LI, SI) for Time.
Method | Mean rank | Sum of ranks | Test statistics | Value |
---|---|---|---|---|
LI | 20.25 | 405.0 | Mann-Whitney |
195 |
SI | 20.75 | 415.0 | Wilcoxon |
405 |
— | — | — |
|
|
Results of Mann-Whitney test on relative errors of the pair (LI, SI) for
Method | Mean rank | Sum of ranks | Test statistics | Value |
---|---|---|---|---|
LI | 13.54 | 176.0 | Mann-Whitney |
84 |
SI | 13.46 | 175.0 | Wilcoxon |
175 |
— | — | — |
|
|
In this section, we investigation the impact of the SQP, on the proposed algorithm. For this purpose, we remove the SQP from Algorithm
The numerical results of the without SQP algorithm, for NOCPs in the Appendix.
Problem no. | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | |
|
1.7296 | 0.0174 | 213.80 | 0.1702 | 0.1130 |
|
0.2390 |
|
1.8606 | 0.2048 |
|
|
|
1.1445 | — | — | — |
|
— | — | — |
|
||||||||||
Problem no. | ||||||||||
11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | |
|
||||||||||
|
4.8816 |
|
0.0153 | 4.5727 |
|
2.3182 |
|
0.0336 | 25.13 | 202.74 |
|
|
|
0.0161 |
|
|
|
1.64 | — | 17.06 | 2.5283 |
Here, a two-phase algorithm was proposed for solving bounded continuous-time NOCPs. In each phase of the algorithm, a MHGA was applied, which performed a local search on offsprings. In first phase, a random initial population of control input values in time nodes was constructed. Next, MHGA started with this population. After phase 1, to achieve more accurate solutions, the number of time nodes was increased. The values of the associated new control inputs were estimated by Linear interpolation (LI) or Spline interpolation (SI), using the curves obtained from phase 1. In addition, to maintain the diversity in the population, some additional individuals were added randomly. Next, in the second phase, MHGA restarted with the new population constructed by above procedure and tried to improve the obtained solutions at the end of phase 1. We implemented our proposed algorithm on 20 well-known benchmark and real world problems; then the results were compared with some recently proposed algorithms. Moreover, two statistical approaches were considered for the comparison of the LI and SI methods and investigation of sensitivity analysis for the MHGA parameters.
The following problems are presented using notation given in ( (VDP) [ (CRP) [ (FFRP) [ (MSNIC) [ (CSTCR) [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [
The authors declare that there is no conflict of interests regarding the publication of this paper.