In view of the traditional numerical method to solve the nonlinear equations exist is sensitive to initial value and the higher accuracy of defects. This paper presents an invasive weed optimization (IWO) algorithm which has population diversity with the heuristic global search of differential evolution (DE) algorithm. In the iterative process, the global exploration ability of invasive weed optimization algorithm provides effective search area for differential evolution; at the same time, the heuristic search ability of differential evolution algorithm provides a reliable guide for invasive weed optimization. Based on the test of several typical nonlinear equations and a circle packing problem, the results show that the differential evolution invasive weed optimization (DEIWO) algorithm has a higher accuracy and speed of convergence, which is an efficient and feasible algorithm for solving nonlinear systems of equations.
Systems of nonlinear equations arise in many domains of practical importance such as engineering, mechanics, medicine, chemistry, and robotics. Solving such a system involves finding all the solutions (there are situations when more than one solution exists) of the polynomial equations contained in the mentioned system. The algorithms of solving nonlinear equations systems are worse than linear equations in convergence speed and ratio, especially solving nonconvex nonlinear equations. The traditional solutions of nonlinear equations include Newton-Raphson method, Quasi-Newton method, and homotopy method. Newton-Raphson method is a much more classical method, but it is sensitive to initial iteration value. In addition, not only it requires a large amount of calculation, but also sometimes it accompanied by difficulty calculation. Quasi-Newton method is to solve the difficult caused by Jacobi matrix. It has now become one of the most effective methods that solve nonlinear equations and optimization problems. While, its stability is poor and sometimes its iterative effect is not well. The basic idea of homotopy method is to start from easily solved equations and then gradually transit to the original equations and get the solution of problems. In recent years, with the rapid development of computational intelligence, computational intelligence techniques have also been used to solve nonlinear equations, such as genetic algorithm [
In 2006, a novel stochastic optimization model, invasive weed optimization (IWO) algorithm [
In this paper, a new hybrid algorithm based on the population diversity of IWO and the heuristic differential evolution (DE) is presented to solve nonlinear systems of equation. With the population diversity, which enhances the global searching ability of algorithm, and heuristic method, which improves the local mining capacity, the hybrid algorithm gets a higher optimization accuracy and faster convergence speed. The performance of the proposed approach is evaluated for several well-known benchmark problems from kinematics, chemistry, combustion, and medicine. Numerical results reveal the efficiency of the proposed approach and its flexibility to solve large-scale systems of equations.
The general form of nonlinear equations systems can be described as follows:
In the basic IWO, weeds represent the feasible solutions of problems and population is the set of all weeds. A finite number of weeds are being dispread over the search area. Every weed produces new weeds depending on its fitness. The generated weeds are randomly distributed over the search space by normally distributed random numbers with a mean with a mean equal to zero equal to zero. This process continues until the maximum number of weeds is reached. Only the weeds with better fitness can survive and produce seed, others are being eliminated. The process continues until maximum iterations are reached or hopefully the weed with the best fitness is closest to the optimal solution. The process is addressed in detail as follows.
A population of initial solutions is being dispread over the
The higher the weed’s fitness is, the more seeds it produces. The formula of weeds producing seeds is
The generated seeds are randomly distributed over the
After passing some iteration, the number of weeds in a colony will reach its maximum (
Reviewing the IWO, some initial weeds are dispread over the search space randomly and then produce new individuals (seeds). We select the better plants from the population consisting of weeds and seeds. The process continues until maximum number of plants is reached. In order to speed up the optimization process, differential evolution (DE) [
BEGIN Input: randomly initialize a population of DEIWO individuals. While (the termination criterion is not satisfied) Calculate every individual’s fitness Calculate the number of every individual’s offspring Update Generate seeds over the search space by Add the generated seeds into the solution set; If (the number of all weeds and seeds equal the maximum population End If Mutate the individual by Cross the individual by Select individual between End While Output the best individual and its fitness; END
Differential evolution (DE), introduced by Storn and Price in 1997, is one of the most prominent new generation EAs for solving real-valued optimization problems. Using only a few control parameters, DE offers exclusive advantages, such as a simple and easy-to-understand concept, ease of implementation, powerful search capability, and robustness. The main procedure of DE includes mutation, crossover, and selection. These operators are based on natural evolution principles in order to keep the population diversity, as well as to avoid premature convergence. In this way, mutation and crossover operators are used to generate new vectors. Then, the selection operator determines which vectors will survive in the next generation. This procedure is repeated until a stopping condition is reached. The basic principle of DE is given as follows.
Suppose the objective function can be expressed as follow.
The individual of the next generation is generated as the follows:
The hybrid strategy of DE and IWO can be summarized as follows. Every initial weed produces new seeds depending on its fitness. The generated seeds are randomly distributed over the search space by normally distributed random numbers. This process continues until the maximum number of weeds is reached. Only the weeds with better fitness can survive and produce seed, when the others are being eliminated. We can see that IWO ensures the diversity of individuals and the worse individuals also have the opportunity to reproduce, while individuals of DE move toward the best individual, using the information of distance and direction obtained from the current population. We can conclude that DE improves the convergence speed and searching precision. So, combining DE and IWO is useful to enrich the search behavior of optimization process and get high quality solutions.
This section reports several experiments and comparisons using the proposed approach. Some well-known applications are also considered in the subsequent section.
The experimental program testing platform included processor: CPU Intel Core i3-370, frequency: 2.40 GHz, memory: 4 GB, operating system: Windows 7, and run software: MATLAB 7.6.
In order to test the performance of DEIWO for solving nonlinear equation systems, 8 nonlinear equation systems in the literature are used and the testing results are compared with the literature [
Consider the following nonlinear system:
The results obtained by applying Newton, Secant, Broyden, and Effati methods; evolutionary approach (EA); Probabilistic-driven search; IWO; and the proposed DEIWO method are presented in Table
Consider the following nonlinear system:
The results obtained by Effati, evolutionary approach (EA), probabilistic-driven search, IWO, and the proposed DEIWO method are given in Table
We consider one benchmark problem proposed from interval arithmetic [
Parameters used by the DEIWO approach are listed in Tables
We considered the example proposed in [
We consider the chemical equilibrium system as given by the following [
We consider the kinematic application kin2 as introduced in [
Coefficients
|
|
|
|
|
---|---|---|---|---|
|
−0.249150680 | +0.125016350 | −0.635550077 | +1.48947730 |
|
+1.609135400 | −0.686607360 | −0.115719920 | +0.23062341 |
|
+0.279423430 | −0.119228120 | −0.666404480 | +1.32810730 |
|
+1.434801600 | −0.719940470 | +0.110362110 | −0.25864503 |
|
+0.000000000 | −0.432419270 | +0.290702030 | +1.16517200 |
|
+0.400263840 | +0.000000000 | +1.258776700 | −0.26908494 |
|
−0.800527680 | +0.000000000 | −0.629388360 | +0.53816987 |
|
+0.000000000 | −0.864838550 | +0.581404060 | +0.58258598 |
|
+0.074052388 | −0.037157270 | +0.195946620 | −0.20816985 |
|
−0.083050031 | +0.035436896 | −1.228034200 | +2.68683200 |
|
−0.386159610 | +0.085383482 | +0.000000000 | −0.69910317 |
|
−0.755266030 | +0.000000000 | −0.079034221 | +0.35744413 |
|
+0.504201680 | −0.039251967 | +0.026387877 | +1.24991170 |
|
−1.091628700 | +0.000000000 | −0.057131430 | +1.46773600 |
|
+0.000000000 | −0.432419270 | −1.162808100 | +1.16517200 |
|
+0.049207290 | +0.000000000 | +1.258776700 | +1.07633970 |
|
+0.049207290 | +0.013873010 | +2.162575000 | −0.69686809 |
Parameters values used by the DEIWO.
|
|
|
|
|
|
|
CR |
---|---|---|---|---|---|---|---|
10 | 15 | 0.01 | 15 | 1 | 4 | 2 | 0.9 |
Benchmarks used in the experiments.
Benchmark | Number of variables | Variables range | Number of iterations |
---|---|---|---|
(1) Interval arithmetic | 10 |
|
2000 |
(2) Neurophysiology application | 6 |
|
800 |
(3) Chemical equilibrium application | 5 |
|
800 |
(4) Kinematic application | 8 |
|
800 |
(5) Combustion application | 10 |
|
800 |
(6) Economics modeling application | 20 |
|
800 |
Comparison of results for Example
Method | Solution | Functions values | Error |
---|---|---|---|
Newton |
|
|
|
Secant |
|
|
|
Broyden |
|
|
|
Effati |
|
|
|
EA [ |
|
|
|
PDS [ |
|
|
|
IWO | |||
500 iterations |
|
|
|
800 iterations |
|
|
|
DEIWO | |||
500 iterations |
|
|
|
800 iterations |
|
|
|
Comparison of results for Example
Method | Solution | Functions values | Error |
---|---|---|---|
Effati |
|
|
|
EA [ |
|
|
|
PDS [ |
|
|
|
IWO | |||
500 iterations |
|
|
|
800 iterations |
|
|
|
DEIWO | |||
500 iterations |
|
|
|
800 iterations |
|
|
|
Comparison of results for interval arithmetic benchmark.
Algorithm | EA | DEIWO | ||||
---|---|---|---|---|---|---|
Solution | Functions values | Error | Solution | Functions values | Error | |
Solution 1 | 0.0464905115 |
0.2077959240 |
0.5578712619 | 0.266267198585396 |
0.008394489588241 |
0.00009717937 |
| ||||||
Solution 2 | 0.1224819761 |
0.1318552790 |
0.4739431843 | 0.259070856110057 |
0.001390867407193 |
0.0000879354 |
| ||||||
Solution 3 | 0.0633944399 |
0.1908436653 |
0.6396343329 | 0.258177973482307 |
0.000279235911393 |
0.0000920236 |
| ||||||
Solution 4 | 0.1939820199 |
0.0603335280 |
1.4462409770 | 0.256723977550822 |
0.000978898430504 |
0.0000663200 |
| ||||||
Solution 5 | 0.1169663983 |
0.1376466161 |
0.7766780224 | 0.255903538082730 |
0.001871285966801 |
0.0000789255 |
| ||||||
Solution 6 | 0.0600624922 |
0.1941520526 |
0.7079949322 | 0.255799163028461 |
0.002038827838389 |
0.0001078856 |
| ||||||
Solution 7 | 0.2077500302 |
0.0464943050 |
0.6818033772 | 0.255930313799156 |
0.001810207471303 |
0.00007961456 |
| ||||||
Solution 8 | −0.0364260444 |
0.2907604740 |
0.6614924105 | 0.258312377722712 |
0.000521318782360 |
0.0000970288 |
Comparison of results for neurophysiology application benchmark.
Algorithm | EA | DEIWO | ||||
---|---|---|---|---|---|---|
Solutions | Functions values | Error | Solutions | Functions values | Error | |
Solution 1 | −0.8282192996 |
0.3139636069 |
0.1207940789 | −0.513317629102986 |
|
0.0000016131 |
| ||||||
Solution 2 | −0.6512719807 |
0.3607740323 |
0.1462019196 | 0.474689101876457 |
|
0.0000007971 |
| ||||||
Solution 3 | 0.0425943625 |
0.1489636110 |
0.1477907295 | 0.509450781724102 |
|
0.0000014868 |
| ||||||
Solution 4 | 0.3269911198 |
0.2710537507 |
0.1340463271 | −0.866626598784898 |
|
0.0000010566 |
| ||||||
Solution 5 | 0.8625703877 |
0.2043564483 |
0.1151261990 | −0.999219226854328 |
|
0.0000006850 |
| ||||||
Solution 6 | 0.7618711576 |
0.1608754444 |
0.0596389890 | 0.855393409271579 |
|
0.0000010225 |
| ||||||
Solution 7 | 0.6609930931 |
0.0837968094 |
0.4711746758 | 0.757633159666954 |
|
0.0000013232 |
| ||||||
Solution 8 | 0.2739187054 |
0.0075566488 |
0.2152949830 | 0.643020583613510 |
|
0.0000011549 |
| ||||||
Solution 9 | −0.0838634907 |
0.2102336348 |
0.1303553523 | −0.943535194739874 |
|
0.0000014553 |
| ||||||
Solution 10 | 0.4612359064 |
0.0889325997 |
0.0495497431 | −0.594047159308489 |
|
0.0000003212 |
| ||||||
Solution 11 | −0.6907741758 |
0.2281557150 |
0.0583912523 | 0.818188591099612 |
|
0.0000001070 |
| ||||||
Solution 12 | −0.8078668904 |
0.0050092197 |
0.0033499526 | 0.728516375898463 |
|
0.0000014555 |
Comparison of results for chemical equilibrium application benchmark.
Algorithm | EA | DEIWO | ||||
---|---|---|---|---|---|---|
Solutions | Functions values | Error | Solutions | Functions | Error | |
Solution 1 | −0.0163087455 |
0.1525772447 |
0.2537179573 | 0.047039050817101 |
0.026769403348726 |
0.0007637704 |
| ||||||
Solution 2 | 0.3357311285 |
0.3491354953 |
0.4345218367 | 0.026474719105915 |
0.009748439199621 |
0.0003118371 |
| ||||||
Solution 3 | 0.3273318676 |
0.3206895328 |
0.3799218328 | 0.043701369103126 |
0.024510445238943 |
0.0007169003 |
| ||||||
Solution 4 | 0.1626252165 |
0.0298611415 |
0.3113039788 | 0.041608285134930 |
0.018067590257058 |
0.0006.929642 |
| ||||||
Solution 5 | 0.3221051215 |
0.3510711865 |
0.2740228596 | 0.041994672912187 |
0.020960255065471 |
0.0007143232 |
| ||||||
Solution 6 | 0.0429158354 |
0.0446360606 |
0.2625846864 | 0.039142614022070 |
0.020928314409356 |
0.0005034014 |
| ||||||
Solution 7 | 0.7276812579 |
0.3738656386 |
0.3273224789 | 0.029777144983889 |
0.012756030203747 |
0.0003990879 |
| ||||||
Solution 8 | 0.1296088399 |
0.0660183067 |
0.1698830307 | 0.034568611394593 |
0.018414022774285 |
0.0004157540 |
| ||||||
Solution 9 | 0.3367208030 |
0.2796624109 |
0.2383566164 | 0.029700290568578 |
0.017030072621489 |
0.0003695853 |
| ||||||
Solution 10 | 0.4224008806 |
0.2712921597 |
0.1772971294 | 0.039076419522626 |
0.014604592480938 |
0.0007996812 |
| ||||||
Solution 11 | 0.2395706253 |
0.2975483261 |
0.3034471293 | 0.029989577399948 |
0.011074163193089 |
0.0004476638 |
| ||||||
Solution 12 | 0.1671662105 |
0.1245889560 |
0.1562964598 | 0.026264550497174 |
0.014873167160221 |
0.0003307343 |
| ||||||
Solution 13 | 0.4586478321 |
0.3386346204 |
0.2353790429 | 0.039763793276441 |
0.020273590576762 |
0.0005721555 |
| ||||||
Solution 14 | 0.4064810686 |
0.3392512646 |
0.2344978766 | 0.042494575252538 |
0.022694195654279 |
0.0006578230 |
Comparison of results for kinematic application benchmark.
Algorithm | EA | DEIWO | ||||
---|---|---|---|---|---|---|
Solutions | Functions values | Error | Solutions | Functions values | Error | |
Solution 1 | −0.0625820337 |
0.3911967824 |
2.8055186567 | 0.045507371706493 |
0.001754595515567 |
0.0001004601 |
| ||||||
Solution 2 | −0.1564353525 |
0.7723864643 |
1.8197999485 | −0.209908402344501 |
0.000822682189556 |
0.0001426091 |
| ||||||
Solution 3 | −0.5618177814 |
0.0336943748 |
1.7384318314 | −0.202282750875349 |
0.002202615985165 |
0.0001181619 |
| ||||||
Solution 4 | 0.0608363294 |
0.7409305497 |
2.3539965598 | 0.089422621197712 |
0.003572640293149 |
0.0000791293 |
| ||||||
Solution 5 | −0.7461742647 |
0.1552859413 |
2.3132511111 | 0.005618623394285 |
0.012038481762346 |
0.0001565654 |
| ||||||
Solution 6 | −0.3064809352 |
0.9060378884 |
3.8308029184 | 0.077779987853934 |
0.000692862137470 |
0.0000905605 |
| ||||||
Solution 7 | −0.5739275815 |
0.5286935399 |
1.9576485695 | 0.062451694147746 |
0.002479214228837 |
0.0000526779 |
| ||||||
Solution 8 | 0.7822939914 |
0.1443884130 |
0.6032920001 | 0.129736374626430 |
0.000932222022973 |
0.0000690265 |
| ||||||
Solution 9 | −0.7461742647 |
0.0762392460 |
1.3773369662 | 0.028424442196598 |
0.004476461376883 |
0.0000566727 |
| ||||||
Solution 10 | −0.6205399028 |
0.1461818771 |
2.5250914998 | 0.270276389119342 |
0.007641330839746 |
0.0001467558 |
We consider the combustion problem for a temperature of 3000°C as proposed in [
Comparison of results for combustion application benchmark.
Algorithm | EA | DEIWO | ||||
---|---|---|---|---|---|---|
Solutions | Functions values | Error | Solutions | Functions values | Error | |
Solution 1 | −0.0552429896 |
0.0274133878 |
0.0371466690 | 0.003672929585885 |
0.002043691593620 |
0.0000304137 |
| ||||||
Solution 2 | −0.0338378558 |
0.0008794626 |
0.0794603736 | 0.000170916648864 |
0.000847476006004 |
0.0000044657 |
| ||||||
Solution 3 | −0.0103209333 |
0.1170689295 |
0.0601546074 | 0.012819319246173 |
0.001067920987079 |
0.0000247769 |
| ||||||
Solution 4 | 0.0177198747 |
0.0262522791 |
0.1058284540 | 0.015469554753745 |
0.003486775086451 |
0.0000195602 |
| ||||||
Solution 5 | 0.0348357700 |
0.0289865877 |
0.0768661970 | −0.001269492482752 |
0.000878131789857 |
0.0000162755 |
| ||||||
Solution 6 | 0.0172348545 |
0.0202656999 |
0.0042879841 | −0.001858395184513 |
0.001820431874319 |
0.0000300866 |
| ||||||
Solution 7 | −0.1443475355 |
0.0460704291 |
0.0134575843 | −0.002441213851907 |
0.001484619742542 |
0.0000291561 |
| ||||||
Solution 8 | 0.1612054472 |
0.0966899590 |
0.0231646270 | −0.013880682509557 |
0.001750147547270 |
0.0000254721 |
Comparison of results for economics modeling application benchmark.
Algorithm | EA | DEIWO | ||||
---|---|---|---|---|---|---|
Solutions | Functions values | Error | Solutions | Functions values | Error | |
Solution 1 | −0.1639324 |
|
0.4094911811 | −6.219110636449961 |
0.000206621288237 |
0.0000004629 |
| ||||||
Solution 2 | −0.2071340 |
0.0050496 |
0.1866532809 | −8.258178201520460 |
0.000112448421209 |
0.0000023157 |
| ||||||
Solution 3 | 0.0936580 |
0.0004909 |
0.2694715119 | 0.410175655605525 |
0.000174067214160 |
0.0000057950 |
| ||||||
Solution 4 | −0.2686292 |
0.0015673 |
0.0336841112 | 0.309460292219735 |
0.000526241325241 |
0.0000017369 |
The following modeling problem is considered as difficult and can be scaled up to arbitrary dimensions [
The constant
The parameters used by the DEIWO approach are presented in Tables
Table
For Example
Results of 500 iterations.
Results of 800 iterations.
Results of 500 iterations.
Results of 800 iterations.
For interval arithmetic benchmark, we can see the errors of all solutions obtained by DEIWO are almost in E-005. Moreover, the given 8 solutions are stable. Phenomenon of fluctuation does not appear. Thus, DEIWO can effectively resolve high-dimensional equation systems.
For neurophysiology application benchmark, 12 solutions are given in Table
Table
For kinematic application benchmark, we can see that the errors of EA are very large, such as Solution 6; the error is 3.8, while the errors of DEIWO are between E-004 and E-005. Solutions for combustion application are given in Table
The number of equations for economics modeling application is variable. In our experiment, the number is 20. As can be seen from the Table
The running time required for our algorithm to converge is presented in Table
Comparison of results for CPU time required by the DEIWO for all the examples.
Example |
Example |
Example |
Example |
Example |
Example |
Example |
Example |
|
---|---|---|---|---|---|---|---|---|
EA [ |
5.14 | 5.09 | 39.07 | 28.90 | 32.71 | 221.09 | 151.12 | 640.92 |
PDS [ |
5.00 | 5.00 | 30 | 30 | 30 | 30 | 30 | 20 |
DEIWO | 2.27 | 2.27 | 15.05 | 13.11 | 12.25 | 16.05 | 17.25 | 18.14 |
Comparison of the number of function evaluations required by the DEIWO for all the examples.
Example |
Example |
Example |
Example |
Example |
Example |
Example |
Example |
|
---|---|---|---|---|---|---|---|---|
EA [ |
1 | 1 | 8 | 12 | 14 | 10 | 8 | 4 |
PDS [ |
1 | 1 | 1 | 2 | 2 | 2 | 2 | 3 |
DEIWO | 2 | 2 | 8 | 12 | 14 | 10 | 8 | 4 |
The geometric constraint problem is generally divided into three categories: integrity constraint, underconstraint, and overconstraint [
Single circle packing problem model is as shown in Figure
Results of solving geometric constraint.
|
Algorithm |
|
|
|
|
|
---|---|---|---|---|---|---|
|
PSO | 10.0000 | 17.3205 | 10.0000 | 5.7735 |
|
DEIWO | 10.0000000215 | 17.3205081063 | 10.0000000126 | 5.7735026797 |
|
|
| ||||||
|
PSO | 6.8750 | 7.2618 | 7.5000 | 3.2275 |
|
DEIWO | 6.8750000954 | 7.2618437417 | 7.5000003760 | 3.2274862446 |
|
|
| ||||||
|
PSO | −0.7677 | 13.5783 | 2.7750 | 2.9363 |
|
DEIWO | 0 | 13.7395290033 | 2.9816440198 | 2.9817236110 | 12.66797 | |
| ||||||
|
PSO | −20.0294 | −0.0169 | −6.2739 | 1.9062 | 303.5985 |
DEIWO | 0 | 10.9711605462 | 4.0800150911 | 4.0797379306 |
|
Single circle packing problem model.
The DEIWO algorithm is very efficient for solving equations systems. The algorithm has the abilities to overcome local optimal solutions and to obtain global optimal solutions.
In this paper, based on the IWO algorithm and DE algorithm, we present a new hybrid DEIWO algorithm for solving nonlinear equation systems. The proposed DEIWO approach seems to be very efficient for solving equations systems. We analyzed the case of nonlinear equations systems. We first compared our approach for some simple equations systems having only two equations that were recently used for analyzing the performance of a newly proposed method. The results obtained using the proposed DEIWO optimization approach are very promising. In optimization process, useful information obtained by DE from population is used to guide evolution direction of weeds. Experiments show that the DEIWO is effective and the proposed method could be extended for more higher dimensional systems, although this will also involve increased computational complexity. In a similar manner, we can also solve inequality systems and systems of differential equations, which are part of our future research work.
This work is supported by the National Science Foundation of China under Grant no. 61165015, Key Project of Guangxi Science Foundation under Grant no. 2012GXNSFDA053028, Key Project of Guangxi High School Science Foundation under Grant no. 20121ZD008, the Funded by Open Research Fund Program of Key Lab of Intelligent Perception and Image Understanding of Ministry of Education of China under Grant no. IPIU01201100, and the Innovation Project of Guangxi Graduate Education under Grant no. YCSZ2012063.