^{1}

^{2}

^{3}

^{3}

^{1}

^{2}

^{3}

This paper introduces an expanded version of the Invasive Weed Optimization algorithm (exIWO) distinguished by the hybrid strategy of the search space exploration proposed by the authors. The algorithm is evaluated by solving three well-known optimization problems: minimization of numerical functions, feature selection, and the Mona Lisa TSP Challenge as one of the instances of the traveling salesman problem. The achieved results are compared with analogous outcomes produced by other optimization methods reported in the literature.

An instance of an optimization problem is a pair

A metaheuristic is a strategy designed to efficiently explore the search space in order to find near-optimal solutions [

The Invasive Weed Optimization (IWO) algorithm is a metaheuristic, in which the exploration strategy of the search space is based on the transformation of a complete solution of the considered problem into another one. The authors of the original version of the algorithm from University of Tehran were inspired by observation of dynamic spreading of weeds and their quick adaptation to environmental conditions. The fundamental components of the algorithm are [

Usefulness of the IWO was confirmed for both continuous and discrete optimization problems. The research was focused

The goal of the authors is to introduce an expanded version of the IWO (“exIWO”) distinguished by the hybrid strategy of the search space exploration proposed by the authors. In addition, the IWO competitive exclusion mechanism was enriched by two variants of individuals selection, which were incorporated into the algorithm. The exIWO was evaluated by solving three optimization problems: minimization of numerical functions, feature selection, and traveling salesman problem.

The organization of the paper is as follows: Section

Similarly to the majority of evolutionary algorithms, the idea of the exIWO can be described by the following difference equation [

The simplified pseudocode mentioned in Algorithm

Create the first population.

Compute the value of the fitness function.

Compute the number of seeds.

Draw the dissemination method

(

Create a new individual according to the chosen method.

Compute the value of its fitness function.

Select individuals for a new population.

Return the best individual.

The main intention of the authors of the present paper was to enrich the IWO’s strategy of the search space exploration with components which allow for enlargement of the analyzed area as well as examination of the local extremum in the vicinity of the current point of the space. Hence, the exIWO makes use of the hybrid “dissemination of seeds” strategy, which is responsible for the “spatial dispersal,” but apart from the “dispersing” method based on the IWO’s competitive exclusion it includes two additional mechanisms: “spreading” and “rolling down.” A flowchart of the exIWO is presented in Figure

Flowchart of the exIWO.

The optimization process starts with a random initialization of the first population. However, a greedy approach or, in general, knowledge of “good” solutions can be also considered while constructing protoplasts of individuals whose features in addition have a chance to be refined in next populations. It is worthwhile to mention that the best solution found by the exIWO cannot be of worse quality than the best one of protoplasts created in a “controlled” manner.

Stop criterion can be defined as the number of populations or as the execution time limit.

In accordance with the IWO the number of seeds

/*

Create randomly a new individual

Return

/*

Determine a degree of difference

offspring being created.

/*

Return

/*

Perform a single transformation of

Compute the value of the neighbour’s fitness function.

Return

The spreading (Algorithm

Idea of (a) spreading, (b) dispersing, (c) rolling down (

The

The rolling down (Algorithm

In continuous optimization problems the distance between the parent plant and the place where the seed falls on the ground, computed by the normal distribution generator constitutes the basis for both the dispersing and the rolling down. Construction of a new individual starts with the random generation of values assigned to particular elements of the structure representing the individual (e.g., arguments of

Candidates for next population are selected in a deterministic manner according to one of the following methods: global, offspring-based, and family-based. Set of candidates for the

Essential differences between IWO and exIWO were collected in Table

Main differences between IWO and exIWO.

Component of the algorithm | IWO | exIWO |
---|---|---|

Initialization of a population | Random | Random or greedy |

Spatial dispersal | Dispersing | Spreading, dispersing, and rolling down |

Competitive exclusion | Global | Global, offspring-based, or family-based |

The goal of the research was to adapt the exIWO metaheuristic for solving three optimization problems: minimization of numerical functions, feature selection, and the Mona Lisa TSP Challenge, to conduct experiments and to compare their results with analogous outcomes produced by other optimization methods reported in the literature. Feature selection evaluated on the basis of classification accuracy belongs to maximization problems, whereas the Mona Lisa TSP Challenge requires minimization of evaluation function.

The workstation used for experiments is described by the following parameters: 2 × Intel Xeon E5620 2.40 GHz RAM 16 GB MS Windows Server 2008 R2 Datacenter 64-bit SP1.

The optimized multidimensional functions: sphere, Griewank, Rastrigin, and Rosenbrock, are frequently used as benchmarks which allow comparing the experimental results with those produced by other algorithms. The minimum values found by the exIWO were confronted with the outcomes generated by the IWO and by the genetic algorithm (GA) as well as with the results of the standard Particle Swarm Optimization (SPSO) reported in the literature. For comparative purposes initial scope of the search space for the exIWO was determined each time by the conditions proposed by the authors of the referenced articles.

The formula defining the

The Griewank function: (a)

The Rastrigin function: (a)

(a) The Rosenbrock function (

An individual is represented by a vector of a length equal to

As was previously explained, in case of minimization problem the formula (

Following the idea of the IWO authors, the convergence of the exIWO was first tested on the basis of two-dimensional sphere function. Results of the experiment are presented in Figure

Convergence of exIWO to the optimum of 2-dimensional sphere function.

Convergence tests of the exIWO were also performed for

The exIWO parameters used for the minimization of numerical functions.

Description | Griewank ( |
Griewank ( |
Rastrigin | Rosenbrock |
---|---|---|---|---|

Population cardinality | {20, 40, 80, 160} | |||

Number of iterations (stop criterion) | {1000, 1500, 2000} | |||

Initialization of the first population | Random | |||

Max. number of seeds for a weed |
5 | 5 | 3 | 6 |

Min. number of seeds for a weed |
0 | 0 | 1 | 0 |

Initial value of standard deviation |
25 | 75 | 25 | 2.5 |

Final value of standard deviation |
0.001 | 0.005 | 0.025 | 0.0075 |

Nonlinear modulation factor |
5 | 4.5 | 3 | 4.75 |

Number |
1 | 1 | 1 | 1 |

Probability of the dispersing |
0.7 | 0.3 | 0.8 | 0.1 |

Probability of the spreading |
0.2 | 0.2 | 0 | 0.1 |

Probability of the rolling down |
0.1 | 0.5 | 0.2 | 0.8 |

Selection method | Global | Global | Global | Family-based |

GA and exIWO convergence plot for the Griewank function.

GA and exIWO convergence plot for the Rosenbrock function.

GA and exIWO (global selection) convergence plot for the Rastrigin function.

The exIWO convergence plot for the Rastrigin function using offspring-based or family-based selection.

It should be noted that the selection operator in the GA uses an elitist strategy according to which a predetermined number of individuals with the best fitness values pass to the next generation. This strategy corresponds to the concept of global selection. In case of the exIWO different variants of selection were tested and the global method turned out to be the most promising strategy for all functions except Rosenbrock. However, differences between global and family-based techniques were slight within the scope of the given function. Dissimilarities between selection strategies are shown in Figures

A comparison of GA and exIWO shows that the latter algorithm converges faster in most examined cases.

Experiments related to the numerical functions minimization were also performed for the purpose of comparison of exIWO and IWO. The authors’ assumption was to retain conditions proposed for IWO in [

The parameters of exIWO and IWO used for minimization of the sphere function.

Description | Value (exIWO) | Value (IWO) |
---|---|---|

Population cardinality | 20 | 20 |

Execution time limit (stop criterion) [s] | 5 | 5 |

Initialization of the first population | Random | Random |

Maximum number of seeds sowed by a weed |
4 | 5 |

Minimum number of seeds sowed by a weed |
0 | 0 |

Initial value of standard deviation |
0.1 | 3 |

Final value of standard deviation |
0.001 | 0.001 |

Nonlinear modulation factor |
10 | 3 |

Number |
1 | — |

Probability of the dispersing |
0.3 | 1 |

Probability of the spreading |
0.3 | 0 |

Probability of the rolling down |
0.4 | 0 |

Selection method | Global | Global |

Comparison between IWO and exIWO based on the sphere function.

The comparative research on IWO and exIWO was carried on using 30-dimensional Griewank and Rastrigin functions. Similarly to the computations related to the sphere function, the IWO parameters which were collected in Table

The parameters of IWO used for minimization of Griewank and Rastrigin functions.

Description | Value |
---|---|

Population cardinality | 20 |

Number of iterations (stop criterion) | {100, 500, 2000, 5000, 10000, 20000} |

Initialization of the first population | Random |

Maximum number of seeds sowed by a weed |
3 |

Minimum number of seeds sowed by a weed |
0 |

Initial value of standard deviation |
10 |

Final value of standard deviation |
0.02 |

Nonlinear modulation factor |
3 |

Selection method | Global |

Averaged minima of IWO and exIWO for the 30-dimensional Griewank function.

Averaged minima of IWO and exIWO for the 30-dimensional Rastrigin function.

The exIWO which makes use of the hybrid strategy of the search space exploration obtains better results in comparison to those generated by the IWO.

Results reported in [

Initial scope of the search space for each argument of particular functions as well as other optimization parameters corresponds with values proposed in [

Minima of the

Comparison between SPSO and exIWO based on the Rastrigin function.

Comparison between SPSO and exIWO based on the Rosenbrock function.

Comparison between SPSO and exIWO based on the Griewank function.

The results obtained by the exIWO turned out to be better than the outcomes of the SPSO.

All aforementioned experiments revealed the usefulness of the exIWO for solving continuous optimization problems. The method can compete with other algorithms. Moreover, the hybrid strategy of the search space exploration turned out to be more effective than the method proposed in the IWO.

According to one of many descriptions of feature (attribute) selection its aim is to choose a subset of features for improving prediction accuracy or decreasing the size of the structure without significantly decreasing prediction accuracy of the classifier built using only the selected features [

The main idea behind the experiments was to test the exIWO ability to find the possibly best subset of features as descriptors of objects subject to recognition:

The Semeion Handwritten Digit Data Set (Semeion Research Center of Sciences of Communication, via Sersale 117, 00128 Rome, Italy; Tattile Via Gaetano Donizetti, 1-3-5, 25030 Mairano (Brescia), Italy.), [

Data sample from the Semeion Handwritten Digit Data Set.

Gait can be captured by two-dimensional video cameras of surveillance systems or by much accurate motion capture (mocap) systems which acquire motion data as a time sequence of poses. In the latter case the movement of an actor wearing a special suit with attached markers is recorded by NIR (Near Infrared) cameras. Positions of the markers in consecutive time instants constitute basis for reconstruction of their 3D coordinates. Gait sequences were recorded in the Human Motion Laboratory (HML) of the Polish-Japanese Institute of Information Technology [

Motion capture recording in the Human Motion Laboratory.

For both problems an individual was represented by a binary vector of a length equal to the initial number of features

An individual underwent a transformation which was a simple binary mutation of a randomly chosen element of the vector.

Each weed, that is, each subset of features constructed by the exIWO was used as a set of data descriptors by the 1NN classifier in the supervised classification process. Thus, the fitness function was equivalent to the classification accuracy expressed by means of the Correct Classification Rate (CCR) which indicated the percentage of correctly classified cases. For comparative purposes feature selection was also performed by means of the genetic algorithm as well as the Best-first method—both implemented in the WEKA software [

The most appropriate values of the exIWO parameters were collected in Table

The parameters of exIWO used for feature selection.

Description | Value (digits) | Value (mocap data) |
---|---|---|

Population cardinality | 100 | 100 |

Number of iterations (stop criterion) | 1000 | 1000 |

Initialization of the first population | Random | Random |

Maximum number of seeds sowed by a weed |
2 | 2 |

Minimum number of seeds sowed by a weed |
2 | 2 |

Initial value of standard deviation |
0.75 | 8.735 |

Final value of standard deviation |
0.01 | 0.01 |

Nonlinear modulation factor |
6.46 | 2.59 |

Number |
2 | 3 |

Probability of the dispersing |
0 | 0.4 |

Probability of the spreading |
0.2 | 0.2 |

Probability of the rolling down |
0.8 | 0.4 |

Selection method | Offspring-based | Any |

Results of the experiments related to the Semeion Handwritten Digit Data Set are presented in Table

Evaluation of the subset selection mechanisms for the Semeion Handwritten Digit Data Set.

Subset selection mechanism | Classification accuracy (%) | Number of features |
---|---|---|

exIWO | 88.61 | 147 |

Genetic algorithm | 88.23 | 138 |

Best-first | 80.89 | 45 |

Results of the experiments on mocap data were collected in Table

Evaluation of the subset selection mechanisms for the mocap data.

Subset selection mechanism | Classification accuracy (%) | Number of features |
---|---|---|

exIWO | 97.69 | 14 |

Genetic algorithm | 97.69 | 13 |

Best-first | 94.80 | 8 |

Efficiency of the exIWO applied for feature selection does not differ significantly from the results obtained by other tested methods.

In February 2009, Robert Bosch created a 100000-city instance of the symmetric traveling salesman problem (TSP) that provides a representation of Leonardo da Vinci’s Mona Lisa as a continuous-line drawing (Figure

Mona Lisa as a continuous-line drawing [

The optimal solution of the TSP is defined as follows [

From among significant concepts related to the form of a single solution it is worthwhile to mention three vector representations proposed in the literature:

The number of seeds for each individual is determined by the formula (

The first population was initialized greedily—for each individual the start city was chosen randomly and the closest city was iteratively added to the tour from among yet unvisited cities.

A single transformation of an individual is based on the

The exIWO parameters were selected experimentally. Table

The exIWO parameters used for the Mona Lisa TSP Challenge.

Description | Value |
---|---|

Population cardinality | 20 |

Number of iterations (stop criterion) | 1000000 |

Initialization of the first population | Greedy |

Maximum number of seeds sowed by a weed |
5 |

Minimum number of seeds sowed by a weed |
1 |

Initial value of standard deviation |
10 |

Final value of standard deviation |
0.01 |

Nonlinear modulation factor |
3 |

Number |
3 |

Probability of the dispersing |
0.8 |

Probability of the spreading |
0 |

Probability of the rolling down |
0.2 |

Selection method | Family-based |

The tour of length 5 919 404 was found by the exIWO after approximately 19.6 days of computation. It turned out to be worse than the best known result (of length 5 757 191) which was found on March 17, 2009 by Yuichi Nagata [

The percentage difference between the length of the current best tour and the best known result for the Mona Lisa TSP Challenge in consecutive iterations.

It is worthwhile to underline that the final result was achieved by the exIWO making use of greedy method of population initialization and family-based selection in combination with elimination of the spreading from the set of dissemination techniques. This approach was expected to gradually improve “nonaccidental” individuals from the first population.

The authors of the present paper expanded the idea behind the original IWO algorithm introducing a hybrid strategy of the search space exploration consisting of spreading, dispersing, and rolling down. On one hand the strategy allows for enlargement of the analyzed area of the search space; on the other hand it enables examination of the local extremum in the vicinity of the current point of the space. In addition, two variants of individuals selection were incorporated into the algorithm:

Results of experiments with both continuous and discrete optimization problems confirmed the versatility of the exIWO; however the adaptation of the metaheuristic for solving the specific problem requires determination of the following components: a representation of a single solution, a method of initialization of the first population, admissible transformations of an individual, a formula of a fitness function, a stop criterion, and a thorough choice of appropriate values of many algorithm parameters.

Because of the time-consuming character of the last operation, future research plans will focus on the adaptive method for tuning of algorithm parameters.

The authors declare that there is no conflict of interests regarding the publication of this paper.

This work was supported by projects BK2013 from the Silesian University of Technology and DEC-2011/01/B/ST6/06988 from the Polish National Science Centre. Tests for minimization of numerical functions and the Mona Lisa TSP Challenge were performed using the computer server funded by the Ministry of Science and Higher Education, Poland, Grant No. N N516 265835. The authors would like to thank Dr. Bożena Małysiak-Mrozek and Dr. Dariusz Mrozek from the Institute of Informatics, Silesian University of Technology, Gliwice, Poland, for the possibility to perform calculations on this computer.