Journal of Applied Mathematics

Volume 2013 (2013), Article ID 653749, 16 pages

http://dx.doi.org/10.1155/2013/653749

Research Article

A Novel Self-Adaptive Harmony Search Algorithm

Kaiping Luo

School of Economics & Management, Beihang University, Beijing 100191, China

Received 19 July 2013; Revised 4 October 2013; Accepted 8 October 2013

Academic Editor: Julián López-Gómez

Copyright © 2013 Kaiping Luo. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract.

The harmony search algorithm is a music-inspired optimization technology and has been successfully applied to diverse scientific and engineering problems. However, like other metaheuristic algorithms, it still faces two difficulties: parameter setting and finding the optimal balance between diversity and intensity in searching. This paper proposes a novel, self-adaptive search mechanism for optimization problems with continuous variables. This new variant can automatically configure the evolutionary parameters in accordance with problem characteristics, such as the scale and the boundaries, and dynamically select evolutionary strategies in accordance with its search performance. The new variant simplifies the parameter setting and efficiently solves all types of optimization problems with continuous variables. Statistical test results show that this variant is considerably robust and outperforms the original harmony search (HS), improved harmony search (IHS), and other self-adaptive variants for large-scale optimization problems and constrained problems.

1. Introduction

Harmony search (HS) is a relatively new metaheuristic optimization algorithm [1] that imitates the improvisation process of Jazz musicians as shown in Figure 1. Each musician is analogous to each decision variable; the audience’s aesthetics are analogous to the objective function; the pitch range of a musical instrument corresponds to the value range of the decision variables; the musicians’ improvisations correspond to local and global search schemes; musical harmony at certain time corresponds to a solution vector at certain iteration. In general, a musician’s improvisation follows three rules [2]: playing any one pitch from his/her memory, playing an adjacent pitch to one pitch from his/her memory, and playing a random pitch from the possible sound range. The HS algorithm mimics these rules and converts them into corresponding evolutionary strategies: choosing any one value from the HS memory (defined as memory considerations), choosing a value adjacent to one from the HS memory (defined as pitch adjustments), and choosing a random value from the possible value range (defined as randomisation). The evolutionary rules in the HS algorithm are effectively directed using the following parameters: harmony memory size (HMS, i.e., the number of solutions in harmony memory), harmony memory considering rate (HMCR, distinctly ), pitch adjusting rate (PAR, distinctly ), fine-tuning bandwidth (Bw, i.e., the adjustment range of a variable), and the number of improvisations (NI, i.e., the maximum number of iterations).

Figure 1: The Analogy between Improvisation and Optimization (from Geem [3]).

The values of these parameters, particularly three key parameters, HMCR, PAR, and Bw, affect the performances of HS. However, parameter setting is problem dependent and difficult, like other evolutionary computational methods, such as genetic algorithms (GA), simulated annealing (SA), particle swarm optimization (PSO), and ant colony optimization (ACO). To alleviate the burden of manually finding the best parameter values, this work proposes a natural and novel approach to parameter setting based on the improvisation practice of musicians.

For any algorithm that involves the operation of searching randomly, the most difficult is to find an appropriate balance between diversity and intensity in the process of searching. when emphasising diversity, it is easy to conduct a slower convergence and even divergence; when emphasising intensity, it is easy to conduct premature convergence and become trapped in a local optimal solution. The HS algorithm is no exception. Thus, this paper also proposes a novel self-adaptive search mechanism.

The rest of this paper is organized as follows. Section 2 presents a concise review of related previous work. Section 3 describes the novel self-adaptive harmony search (NSHS) algorithm. Section 4 introduces a test suite composed of twelve benchmark functions. Section 5 analyzes the robustness of the proposed variant. Section 6 confirms the superiority of the novel variant by the comparison test. The last section makes a conclusion.

2. Review of Related Previous Work

2.1. Original Harmony Search Algorithm

The original HS algorithm developed by [1] is carried out as follows.

Step 1 (initialise the problem and algorithm parameters). First, the optimization problem is specified as follows:

where is the objective function, is a vector consisting of decision variables, and and are the lower and upper bounds of variable , respectively. The original HS algorithm basically considers the objective function only. If the problem has equality and/or inequality constraints and a solution vector violates any of them, either the resulting infeasible solution will be abandoned or its objective function value will have a certain penalty score added.

Second, the parameters HMS, HMCR, PAR, Bw, and NI must be valued, and the initial harmony memory (HM) must be generated. Consider

Step 2 (improve a new harmony vector by improvisation). A new solution , termed the harmony vector, is generated based on memory considerations, pitch adjustments, and randomisation. Consider

Step 3 (assess the new harmony vector and update the harmony memory). In this step, the merit of the new harmony vector is assessed based on its objective function value. If the new vector is better than the worst harmony vector in the current HM, the new vector is included in the HM and the existing worst vector is excluded from the HM.

Step 4 (check the stopping criterion). Repeat Steps 2~3 until the termination criterion is satisfied; that is, the number of iterations reaches NI.

The structure of the HS algorithm is much easier. Its core evolutionary process includes only three operations: choosing, adjusting, and randomising. However, the meta-heuristic algorithm handles intensification and diversification. Its intensification and diversification are represented by HMCR and PAR, respectively, and its superiority is confirmed by the existence of a large number of successful applications to diverse scientific and engineering problems, including timetabling [4], the 0-1 knapsack problem [5], power system design [6], transmission network planning [7], flow-shop scheduling [8], structural design [9], water network design [10], soil stability analysis [11], ecological conservation [12], and vehicle routing [13].

However, the major disadvantage of the HS algorithm is that its performance depends on the parameter setting and static parameter values are not conducive to balancing intensification and diversification.

2.2. Variants Based on Parameter Setting

To improve the performance of the original HS algorithm or alleviate the burden of manually finding the best parameter values, some variants based on parameter setting have been proposed.

2.2.1. Improved Harmony Search (IHS) Proposed by Mahdavi et al. [14]

In this variant, PAR and Bw are dynamically adjusted by the following formulae:

where and indicate the minimum and maximum pitch adjusting rates, respectively. and NI denote the th-iteration and the maximum number of iterations, respectively. , and indicate the minimum and maximum bandwidths.

In fact, this variant does not alleviate the difficulty of parameter setting, because users have to provide suitable values of , , and . Furthermore, using (4), the pitch adjusting rate gradually increases from to through an iterative process, which is rather inconsistent with the observation that a larger iteration number usually generates a more perfect harmony and requires a smaller tuning probability.

Later, Omran and Mahdavi [15] abandoned the Bw parameter by replacing direct adjustment with the use of a random variable for the best harmony vector as shown in (6), calling this new variant the global-best harmony search (GHS). Consider

As Wang and Huang [16] noted, the name of this variant is misused and the method can easily cause premature convergence. In fact, the most serious defect in the GHS variant is that if the domain of each variable is different, that is, , then the infeasible solution will frequently be generated. Thus, Pan et al. [17] corrected this serious drawback using the following equation:

2.2.2. The Self-Adaptive Harmony Search Proposed by Wang and Huang [16]

Wang and Huang recognised the shortcomings of IHS and GHS. They used the same modification to dynamically adapt PAR values during the search process, but the value of PAR is decreased linearly with time. Thus, their variant needs are still the values of and . The innovation of the variant is to adjust the value of each variable according to the maximal and minimal values in the harmony memory (HM) and without the Bw parameter again. Consider

Although this adjustment enables each variable to avoid violating its boundary constraint, the bandwidth is larger; therefore, the convergence is very slow and can even stop.

2.2.3. The Self-Adaptive Harmony Search Proposed by Pan et al

Pan et al. [18] proposed a local-best harmony search algorithm with dynamic subpopulations (DLHS). In the variant, the whole harmony memory is equally divided into m subgroups. Moreover, to keep population diversity, the small-sized sub-HMs are regrouped once the period is reached. In the variant, HMCR and PAR are from a parameter set list (PSL) with length Len, which is randomly generated by the initial value with a uniform distribution in the range and and then updated once the list is empty. In the variant, the value of the parameter Bw is also dynamically changed:

Although the variant takes the advantage of each local optimum, it consumes too much time as a cost. Thus, Pan et al. [17] proposed a self-adaptive global best harmony search algorithm (SGHS) based on the GHS algorithm. In the new variant, the value of HMCR (PAR) is normally distributed in the range with mean HMCRm (PARm) and standard deviation 0.01 (0.05). The initial values of HMCRm and PARm are given by the user. In a fixed learning period (LP), the values of HMCRm and PARm do not change; after LP, the values are recalculated by averaging all HMCR and PAR values recorded during the previous LP. In the variant, Bw is still determined by (9).

Obviously, the two variants require extra jobs to set the values of more parameters than the basic HS algorithm and other variants. Moreover, it is irrational that the value of PAR still varies in the interval regardless of the number of iterations because a successful search should gradually converge and the value of PAR should be decreased with time to prevent oscillation.

These defects inspired me to investigate how the key parameters of the HS algorithm do not depend on the problem’s feature again.

2.3. Variants Based on Hybridisation with Other Metaheuristics

To improve the performance of the original HS, many researchers have hybridised the algorithm with other meta-heuristics, such as GA, PSO, and ACO [19]. This hybridisation can be categorised into two approaches [20]: integrating some components of other meta-heuristic algorithms into the HS structure, and conversely, integrating some HS components into other meta-heuristic algorithm structures. Perhaps hybrid methods are better than single meta-heuristics.

However, in my opinion, hybridisation will depart from the essential nature of the original heuristic and the resulting mimicry will also be nondescript. Thus, the correct way to investigate this should be by refining the essence of the imitated objects. This idea inspired me to improve the search mechanism of the original HS algorithm by capturing the deep essence of musical improvisation.

3. A Novel Self-Adaptive Harmony Search Algorithm

3.1. Design of the Control Parameters

The HS algorithm includes five parameters: harmony memory size (HMS), harmony memory considering rate (HMCR), pitch adjusting rate (PAR), fine-tuning bandwidth (Bw), and the number of improvisations (NI). Among these parameters, HMCR, PAR, and Bw are crucial for the convergence of HS.

It should be noted that each improvisation of a musician is always accompanied by fine-tuning until the improvisation is finished. That is, it is necessary to adjust each variable value before the HS algorithm is ended. Therefore, it is reasonable that the value of PAR ought to equal one and senseless to consider this parameter again. For this reason, the proposed self-adaptive variant does not use the PAR parameter.

The value of HMCR in the basic HS algorithm is fixed at its initialisation and does not change until the search ends. As the value of HMCR decreases, harmony memory is used less efficiently, the algorithm converges much more slowly, and more time is consumed. As the value of HMCR increases, harmony memory is used more efficiently, the method converges more rapidly, less time is consumed, and the HS is easily trapped in a local optimum. To enhance the power of the HS algorithm, HMCR is usually valued in the interval . However, the reason for this is not revealed [21]. In fact, musicians depend on their memory more for intricate than simple music. This fact inspired me to construct the HMCR parameter based on the dimension of the problem to be solved. That is, the dimension of the problem is analogous to the complexity of the music,

where denotes the dimension of the problem to be solved. Clearly, 0.5 < HMCR < 1; as the number of dimensions of the problem increases, the value of HMCR increases and harmony memory is used more frequently, and vice versa.

It should be noted that musicians fine-tune their playing throughout their improvisation, and the tuning range is wider at the beginning and narrower as the music is completed. This phenomenon inspired me to construct a dynamic fine-tuning bandwidth for each variable , with bounds and and iteration number :

where denotes the standard deviation for the objective function values in the current HM. The value of decreases gradually as the iteration number increases and increases as the range of each decision variable increases. The domain of each decision variable corresponds to the pitch range of each musical instrument; the gradual decrease in the adjustment range as the iteration number increases corresponds to the gradual decrease of fine-tuning bandwidth with increasing improvisation number. is analogous to the unit of fine-tuning bandwidth, and 0.0001 represents the minimal adjustment. represents the slight difference between the merit of each harmony vector in HM. In fact, at the late stages of musical improvisation, the music is gradually mellowed and the fine-tuning range used is very narrow.

3.2. Design of the Self-Adaptive Mechanism

For every meta-heuristic algorithm, accelerating the convergence and preventing the premature convergence appear contradictory. To resolve this contradiction, the evolutionary mechanism of the HS algorithm has to be properly revised. It should be again noted that fine-tuning occurs throughout the whole process of musician improvisation and the fine-tuning range is wider at the early stage and narrower at later stages. Therefore, this phenomenon can be mimicked by the following evolutionary operations:

where random numbers . The standard deviation for the objective function values fstd represents the whole merit of the new harmony vectors in the current HM. If the standard deviation is less than a given threshold 0.0001, that is, if , then the difference between the function values is very small. If the iteration number is sufficiently large, then the distribution of the harmony vectors is centralised; thus, a new random value should be generated within the narrow range for rapid convergence.

3.3. Evolutionary Process of the NSHS Algorithm

On the basis of the novel parameter-setting method and self-adaptive evolutionary mechanism, the computational procedure of the proposed variant can be given, as shown in Algorithm 1.

NSHS Algorithm

Initialise the problem and algorithm parameters:

      Define a standard fitness function by (1)

      Define the parameters HMCR by (10), Bw by (11), HMS and NI

      Generate randomly an initial HM and calculate fstd

Search the best solution:

While () do

        Generate a new harmony vector by the following loop

        While () do

                First, generate a new value for every variable by the following rules:

                If (), then choose uniformly a value from the current memory for variable

                Else, generate randomly

                       If , then generate randomly in the range of

                       Else, generate randomly in the range of

                       End if

                End if

                Second, adjust the new value of every variable by the following rules:

                If , then

                Else, generate randomly in the range of

                End if

        End while

        Assess the merit of the new harmony vector and update the current harmony memory

        If , then

                Accept the new harmony vector and replace the worst vector in the HM with it.

                Recalculate the standard deviation of the fitness function values fstd.

        End if

End while

Determine the optimal solution in the HM by the minimum fitness.

Algorithm 1: Pseudo code of the NSHS algorithm.

4. Benchmark Functions

This section lists the global minimization benchmark functions used to evaluate the performances of the proposed algorithm. The test suite comprises six well-known unimodal benchmark functions, six well-known multimodal benchmark functions, and six constrained optimization functions. To evaluate the robustness of a stochastic search algorithm, the twelve unconstrained benchmark functions are randomly shifted and orthogonally rotated, and four benchmark functions are with normal noise in fitness.

4.1. Unimodal Benchmark Functions

Table 1 summarizes the shifted rotated unimodal benchmark functions. Figure 2 depicts their 3D shapes if the dimension is equal to two.

Table 1: Six shifted rotated unimodal benchmark functions.

Name
Expression

Shifted rotated Cigar-Tablet's function

Shifted rotated Rosenbrock's function

Shifted rotated Schwefel's problem 1.2 with noise in fitness

Shifted rotated Schwefel’s problem 2.21 with noise in fitness

Shifted rotated Schwefel’s problem 2.22 with noise in fitness

Shifted rotated sphere function with noise in fitness

. is a solution in the search range of −100, 100. is a random vector in the range of −100, 100. M is an identity orthogonal matrix. is also a random number. In this study, these biases are randomly given. They are equal to −140, 390, −450, −310, −180, and −450, respectively. For the second benchmark function, global optimum and ; for other functions, global optimum and .

(a) 2D Cigar-Tablet's function

(b) 2D Rosenbrock's function

(c) 2D Schwefel's problem 1.2

(d) 2D Schwefel’s problem 2.21

(e) 2D Schwefel’s problem 2.22

(f) 2D Sphere function

Figure 2: Six well-known unimodal benchmark function landscapes.

4.2. Multimodal Benchmark Functions

Table 2 summarizes the shifted rotated multimodal benchmark functions. Figure 3 depicts their 3D shapes if the dimension is equal to two.

Table 2: Six well-known multimodal benchmark functions.

Name
Expression

Shifted rotated Ackley's function

Shifted rotated Bohachevsky's function

Shifted rotated Griewank's function

Shifted rotated Rastrigin's function

Shifted rotated Schaffer's function

Shifted rotated Schwefel’s problem 2.13

. is a solution in the search range of −10, 10. is a random vector in the range of −10, 10. M is an identity orthogonal matrix. is also a random number. In this study, these biases are randomly given. They are equal to −140, −310, −180, −330, 390, and −460, respectively. For the twelfth benchmark function, global optimum and ; for other functions, global optimum and .

(a) 2D Ackley's function

(b) 2D Bohachevsky's function

(c) 2D Griewank's function

(d) 2D Rastrigin's function

(e) 2D Schaffer's function

(f) 2D Schwefel’s problem 2.13

Figure 3: Six well-known multimodal benchmark function landscapes.

4.3. Constrained Benchmark Functions

The six constrained optimization functions have also been described in the literature [2]. Thus, we only give their corresponding penalty functions defined in this study. It should be noted that these penalty functions are composed of many penalty terms in order to precisely control every violation:

If , then .

If , then .

If , then .

For , the known best solution and . Consider

If , then .

If , then .

If , then .

If , then .

For , the known best solution and . Consider

If , then .

If , then .

If , then .

If , then .

If , then .

If , then .

If , then .

If , then .

If , then .

If , then .

If then .

If , then .

For , the known best solution and . Consider

If , then .

If , then .

If , then .

If , then .

If , then .

For , the known best solution , and . Consider

If , then .

If , then .

If , then .

If , then .

If , then .

If , then .

If , then .

If , then .

If , then .

If , then .

If , then .

If , then .

For , the known best solution , and . Consider

If , then .

If , then .

If , then .

If , then .

If , then .

If , then .

If , then .

If , then .

If , then .

For , the known best solution , , and .

5. Robustness of the Proposed NSHS Algorithm

The performance of a stochastic search algorithm often is affected by the initial populations. For the HS algorithm, the initial population distribution is determined by the value of HMS and the initial search ranges. Thus, this section investigates the robustness of the proposed variant from the two aspects.

5.1. Effect of HMS

Yang [21] declared that the value of HMS is insensitive to the performance of the original HS algorithm. To reveal the fact that the value of HMS does not also have a significant impact on the performance of the NSHS, a full-factorial experiment was conducted. In the experiment, the HMS is valued at four levels (5, 10, 20, and 50) as a fixed factor; above twelve unconstrained benchmark functions with 10-dimension and six constrained functions were employed; the simulation was repeated 30 times, and each simulation underwent 50,000 evaluations for the objective function. The relative error between the running result and the global or known optimum is tested as a dependent variable. The descriptive statistical result of these errors is shown in Table 3.

Table 3: The descriptive statistical result of the relative errors of the NSHS algorithm in the case of HMS = 5, 10, and 50, respectively.

Function
HMS = 5
HMS = 10
HMS = 20
HMS = 50

Mean
Std.
Mean
Std.
Mean
Std.
Mean
Std.

f₁

f₂

f₃

f₄

f₅

f₆

f₇

f₈

f₉

f₁₀

f₁₁

f₁₂

f₁₃

f₁₄

f₁₅

f₁₆

f₁₇

f₁₈

Total

Each group has 30 samples and the experiment has totally 4 * 18 * 30 = 2160 samples.

Clearly, there exists the tiny difference. However, we can claim that the difference is due to the function tested rather than the HMS by the test result of univariate analysis of variance, seeing the report of SPSS shown in Table 4. Therefore, the performance of the proposed NSHS algorithm is not affected by the value of HMS. To save storage space, it is encouraged to use a small HMS, as Pan et al. [17] explained that “since HM is emulating musician’s short-term memory and the short-term memory of the human is known to be small, it is logical.”

Table 4: The report of SPSS on the univariate (relative error) analysis of variance with two fixed factors (problem/function and HMS).

(a) Tests of between-subjects effects (dependent variable: relative error).

Source
Type III sum of squares
df
Mean square

Sig.

Corrected
3.958a
71
.056
11.537
.000

Model

.301
1
.301
62.211
.000

HMS
.038
3
.013
2.632
.048

Problem
3.293
17
.194
40.092
.000

HMS * Problem
.627
51
.012
2.542
.000

Error
10.089
2088
.005

Total
14.347
2160

Corrected

Total
14.046
2159

  squared = .282 (adjusted squared = .257).

(b) Hochberg's multiple comparisons (dependent variable: relative error).

(I) HMS
(J) HMS
Mean difference (I − J)
Std.
Sig.

5
10
.0086
.0042
.225

20
.0100
.0042
.101

50
.0102
.0042
.095

10
5
−.0086
.0042
.225

20
.0014
.0042
1.000

50
.0015
.0042
1.000

20
5
−.0100
.0042
.101

10
−.0014
.0042
1.000

50
.0001
.0042
1.000

50
5
−.0102
.0042
.095

10
−.0015
.0042
1.000

20
−.0001
.0042
1.000

Based on observed means.

5.2. Effect of Initial Search Ranges

To identify whether the performance of the proposed variant depends on the initial HM, a full-factorial experiment was conducted using the parameters HMS = 10 and NI = 50,000. In the experiment, the initial HM was randomly generated at three types of the initial search ranges, that is, the whole domain, the left part, and the right part. The experiment was repeated 30 times and 1620 observations were collected. The descriptive statistical result of the relative errors of the proposed variant is shown in Table 5.

Table 5: The descriptive statistical result of the relative errors of the NSHS algorithm using three types of the initial search ranges.

Function
The whole domain
The left part
The right part

Mean
Std.
Mean
Std.
Mean
Std.

f₁

f₂

f₃

f₄

f₅

f₆

f₇

f₈

f₉

f₁₀

f₁₁

f₁₂

f₁₃

f₁₄

f₁₅

f₁₆

f₁₇

f₁₈

Total

Each group has 30 samples and the experiment has totally 3 * 18 * 30 = 1620 samples.

Obviously, there still exists the small difference. However, the test report of SPSS shown in Table 6 still indicates that the difference is due to the function tested rather than the initial search ranges.

Table 6: The report of SPSS on the univariate (relative error) analysis of variance with two fixed factors (problem/function and initial range).

(a) Tests of between-subjects effects (dependent variable: relative error).

Source
Sum of squares
df
Mean square

Sig.

Corrected
31.122a
53
.587
2.939
.000

Model

1.584
1
1.584
7.927
.005

Initial range
.422
2
.211
1.055
.348

Problem
23.416
17
1.377
6.895
.000

Initial range * problem
7.285
34
.214
1.073
.357

Error
312.834
1566
.200

Total
345.540
1620

Corrected
343.956
1619

  squared = .090 (adjusted squared = .060).

(b) Hochberg's multiple comparisons (dependent variable: relative error).

(I) Initial range
(J) Initial range
Mean difference (I − J)
Std.
Sig.

Left
Right
.0296
.0272
.622

Whole
−.0079
.0272
.988

Right
Left
−.0296
.0272
.622

Whole
−.0375
.0272
.424

Whole
Left
.0079
.0272
.988

Right
.0375
.0272
.424

Based on observed means.

By the results of analysis of variance as shown in Tables 4 and 6, the performance of the proposed NSHS algorithm does not depend on the initial HM. By the descriptive statistical results as shown in Tables 3 and 5, moreover, no matter how the initial HM is, the NSHS algorithm can stability search for the global optima of these benchmark functions with a tiny error, except for function f₂. Therefore, the novel variant is considerably robust.

6. Superiority of the Proposed NSHS Algorithm

To confirm the significant superiority of the proposed variant, a comparison test was conducted. The test compared the performance of the NSHS with the original HS algorithm [1] and the other variants mentioned in Section 2, including Mahdavi’s IHS [14], Wang’s self-adaptive variant represented by HSw for convenience [16], Pan’s DLHS [18], and SGHS [17]. For fair comparison, the parameter settings are those proposed by their developers, as shown in Table 7.

Table 7: Algorithm parameters.

Algorithm
HMS
HMCR
PAR
Bw
NI
Others

HS
9
0.9
0.3
0.01
50,000
None

IHS
9
0.9
Min: 0.01; max: 0.99
Min: 0.0001; max: 1/(20(ub − lb))
50,000
None

HSw
9
0.9
Min: 0.01; max: 0.99
Needless
50,000
None

DLHS
9
Initial: 0.91
Initail: 0.5
Min: 0.0001; max: (ub − lb)/200
50,000
; : len = 200

SGHS
9
Initial: 0.98
Initial: 0.9
Min: 0.0005; max: (ub − lb)/10
50,000
LP = 100

NSHS*
9
1 − 1/()
Needless
No initial value, using (11)
50,000
None

NSHS has only two key parameters HMCR and Bw, but they do not require the user's inputs.

For sufficiently persuasive comparative results, every algorithm was run independently 30 times to solve the aforementioned 18 benchmark functions on the same computer (CPU: AMD 4 × 3.5 Ghz; RAM: 4 GB), and, at the beginning of solving a problem, each algorithm used the same initial HM. The conclusion is made based on an analysis of variance rather than a pairwise -test because Garc’s study yet indicated that the latter method is suitable for a single-problem analysis and a multiple-problem analysis should employ a nonparametric test [22]. Next, the experimental results are elaborated from two aspects, respectively.

6.1. Unconstrained Benchmark Function

The performances of these algorithms regarding twelve unconstrained benchmark functions were investigated at different scales, that is, 10, 100, and 1000. Table 8, Table 10 and Table 12 respectively summarize the descriptive statistical results at different scales. For 10-dimension benchmark functions, the superiority of the NSHS is not significant except for functions f₃ and f₄, as shown in Table 8. The test result shown in Table 9 indicates that the performance of the NSHS is significant better than the IHS at the 95% confidence level and the average error of the NSHS is less than other algorithms except for the DLHS. For 100-dimension benchmark functions, the average error of the NSHS is much less than other algorithms except for the SGHS, as shown in Table 11. Based on the data in Tables 12 and 13, we can see that the superiority of the proposed variant is extremely significant for 1000-dimension benchmark functions.

Table 8: The descriptive statistical result of the relative errors of the six algorithms to solve twelve unconstrained benchmark functions with 10 dimensions.

Function
HS
IHS
HSw
DLHS
SGHS
NSHS

Mean (std.)
Mean (std.)
Mean (std.)
Mean (std.)
Mean (std.)
Mean (std.)

f₁

f₂

f₃

f₄

f₅

f₆

f₇

f₈

f₉

f₁₀

f₁₁

f₁₂

Total

Each group has 30 samples and the experiment has totally 12 * 6 * 30 = 2160 samples. The bold represents the significant best.

Table 9: Dunnett’s one-tailed (>) multiple comparisons between the NSHS and other five algorithms based on the samples of solving twelve unconstrained benchmark functions with 10 dimensions.

(I) Algorithm
(J) Algorithm
Mean difference (I − J)
Std.
Sig.

HS
NSHS
.03782732242345
.0199
.103

IHS
NSHS
.07017638607941*
.0199
.001

HSw
NSHS
.04233406887345
.0199
.064

DLHS
NSHS
−.02008968262494
.0199
.986

SGHS
NSHS
.01957666996786
.0199
.421

The mean difference is significant at the 0.05 level. The test is based on observed means of the dependent variable (relative error).

Table 10: The descriptive statistical result of the relative errors of the six algorithms to solve twelve unconstrained benchmark functions with 100 dimensions.

Function
HS
IHS
HSw
DLHS
SGHS
NSHS

Mean (std.)
Mean (std.)
Mean (std.)
Mean (std.)
Mean (std.)
Mean (std.)

f₁

f₂

f₃

f₄

f₅

f₆

f₇

f₈

f₉

f₁₀

f₁₁

f₁₂

Total

Each group has 30 samples and the experiment has totally 12 * 6 * 30 = 2160 samples. The bold represents the significant best.

Table 11: Dunnett’s one-tailed (>) multiple comparisons between the NSHS and other five algorithms based on the samples of solving twelve unconstrained benchmark functions with 100 dimensions.

(I) Algorithm
(J) Algorithm
Mean difference (I − J)
Std.
Sig.

HS
NSHS
1743032079.82376
72262361658.159
.826

IHS
NSHS
125173553098.6880
72262361658.159
.143

HSw
NSHS
466806.998460054
72262361658.159
.833

DLHS
NSHS
184496.101783522
72262361658.159
.833

SGHS
NSHS
1.39707526033672
72262361658.159
.833

The mean difference is significant at the 0.05 level. The test is based on observed means of the dependent variable (relative error).

Table 12: The descriptive statistical result of the relative errors of the six algorithms to solve twelve unconstrained benchmark functions with 1000 dimensions.

Function
HS
IHS
HSw
DLHS
SGHS
NSHS

Mean (std.)
Mean (std.)
Mean (std.)
Mean (std.)
Mean (std.)
Mean (std.)

f₁

f₂

f₃

f₄

f₅

f₆

f₇

f₈

f₉

f₁₀

f₁₁

f₁₂

Total

Each group has 30 samples and the experiment has totally 12 * 6 * 30 = 2160 samples. For a solvability, the search ranges of function f5 are renewed in −1, 1because 100∧1000 reaches an infinity even 10∧1000. The bold represents the significant best.

Table 13: Dunnett’s one-tailed (>) multiple comparisons between the NSHS and other five algorithms based on the samples of solving twelve unconstrained benchmark functions with 1000 dimensions.

(I) Algorithm
(J) Algorithm
Mean difference (I − J)
Std.
Sig.

HS
NSHS
528495067.3354575 *
7295157.814
.000

IHS
NSHS
533757007.9854518 *
7295157.814
.000

HSw
NSHS
536111617.63627493 *
7295157.814
.000

DLHS
NSHS
525985111.0408457 *
7295157.814
.000

SGHS
NSHS
4198894.56625653
7295157.814
.611

The mean difference is significant at the 0.05 level. The test is based on observed means of the dependent variable (relative error).

6.2. Constrained Benchmark Function

The performances of these algorithms with respect to the aforementioned optimization problems with constraints were investigated. The descriptive statistical result and the report on multiple comparisons are, respectively, shown in Tables 14 and 15. Clearly, the performance of the proposed variant is vastly superior to the HS, the IHS, the HSw, and the SGHS. Although the performance of the NSHS is slightly better than the DLHS, the evolutionary mechanism of the former is simpler and the NSHS does not require too much key-parameters setting. Moreover, the performance of the NSHS is significantly better than the DLHS for functions f₁₅ and f₁₈, seeing Table 14.

Table 14: The descriptive statistical result of the relative errors of the six algorithms to solve six constrained benchmark functions.

Function
HS
IHS
HSw
DLHS
SGHS
NSHS

Mean (std.)
Mean (std.)
Mean (std.)
Mean (std.)
Mean (std.)
Mean (std.)

f₁₃

f₁₄

f₁₅

f₁₆

f₁₇

f₁₈

Total

The experiment collected 6 * 6 * 30 = 1080 observations, 23 out of whom are invalid. For function f₁₅, the DLHS got 2 infeasible solutions. For function f₁₇, the HS got 4 infeasible solutions and the IHS 5, the HSw 3, the DLHS 1, the SGHS 5, and the NSHS 3, respectively. The bold represents the significant best.

Table 15: Dunnett’s one-tailed (>) multiple comparisons between the NSHS and other five algorithms based on the samples of solving six constrained benchmark functions.

(I) Algorithm
(J) Algorithm
Mean difference (I − J)
Std.
Sig.

HS
NSHS
.11043867790257 *
.028
.000

IHS
NSHS
.24772610041475 *
.028
.000

HSw
NSHS
.20691985495254 *
.028
.000

DLHS
NSHS
.01430413405475
.028
.640

SGHS
NSHS
.06953695639486 *
.028
.028

The mean difference is significant at the 0.05 level. The test is based on observed means of the dependent variable (relative error).

7. Conclusion

This paper presented a novel self-adaptive harmony search algorithm (NSHS) for optimization problems with continuous variables with the aim of improving the performance of the harmony search. To investigate the performance of the novel variant, many comparison experiments were conducted based on a test suite comprising six unimodal benchmark functions, six multimodal benchmark functions, and six constrained benchmark functions. Using nonparametric statistical tests, the NSHS’s robustness and superiority were detected. Experimental results on the effect of HMS and initial search ranges indicated that the performance of the NSHS did not depend on the initial HM. The results, when compared with related previous work, manifested that the NSHS performed dramatically better than other six algorithms for large-scale problems and constrained problems.

The originality of this work is mainly reflected in two aspects.

(i)Proposed a simpler approach to the setting of key parameters based on musical practices. The proposed variant has only two key parameters HMCR and Bw, but they do not require the user’s inputs. Moreover, the parameters settings are in accordance with the features of the optimization problem to be solved, such as the scale and the boundaries. Moreover, this approach discards the PAR parameter and further simplifies the parameter setting. Musical improvisation is usually accompanied by fine-tuning; thus, it is senseless to consider this parameter and fine-tuning bandwidth at the same time.(ii)Developed a novel self-adaptive evolutionary scheme based on the standard deviation of the fitness function values in the memory to accelerate algorithm convergence and assure the reaching of a global optimum. This scheme automatically narrows the search space based on a suitable threshold and, meanwhile, solves the problem of the optimal balance between search diversity and intensity.

Acknowledgments

Many thanks to the anonymous reviewer for valuable comments that helped to improve this paper. This work was partly supported by the National Natural Science Foundation of China under Grant 71101003, 91224007, and 71332003 and the Fundamental Research Funds for the Central Universities under Grant YWF-13-D2-JC-12.

References

Z. W. Geem, J. H. Kim, and G. V. Loganathan, “A new heuristic optimization algorithm: harmony search,” Simulation, vol. 76, no. 2, pp. 60–68, 2001.

K. S. Lee and Z. W. Geem, “A new meta-heuristic algorithm for continuous engineering optimization: harmony search theory and practice,” Computer Methods in Applied Mechanics and Engineering, vol. 194, no. 36–38, pp. 3902–3933, 2005.

Z. W. Geem, “State-of-the-art in the structure of harmony search algorithm,” in Studies in Computational Intelligence, Z. W. Geem, Ed., pp. 1–10, Springer, Berlin, Germany, 2010.

M. A. Al-Betar and A. T. Khader, “A harmony search algorithm for university course timetabling,” Annals of Operations Research, vol. 194, pp. 3–31, 2012.

D. Zou, L. Gao, S. Li, and J. Wu, “Solving 0-1 knapsack problem by a novel global harmony search algorithm,” Applied Soft Computing Journal, vol. 11, no. 2, pp. 1556–1564, 2011.

S. Sivasubramani and K. S. Swarup, “Multi-objective harmony search algorithm for optimal power flow problem,” International Journal of Electrical Power and Energy Systems, vol. 33, no. 3, pp. 745–752, 2011.

A. Verma, B. K. Panigrahi, and P. R. Bijwe, “Harmony search algorithm for transmission network expansion planning,” IET Generation, Transmission and Distribution, vol. 4, no. 6, Article ID IGTDAW000004000006000663000001, pp. 663–673, 2010.

L. Wang, Q.-K. Pan, and M. F. Tasgetiren, “Minimizing the total flow time in a flow shop with blocking by using hybridm harmony search algorithms,” Expert Systems with Applications, vol. 37, no. 12, pp. 7929–7936, 2010.

M. P. Saka, “Optimum design of steel sway frames to BS5950 using harmony search algorithm,” Journal of Constructional Steel Research, vol. 65, no. 1, pp. 36–43, 2009.

Z. W. Geem, “Particle-swarm harmony search for water network design,” Engineering Optimization, vol. 41, no. 4, pp. 297–311, 2009.

Y. M. Cheng, L. Li, T. Lansivaara, S. C. Chi, and Y. J. Sun, “An improved harmony search minimization algorithm using different slip surface generation methods for slope stability analysis,” Engineering Optimization, vol. 40, no. 2, pp. 95–115, 2008.

Z. W. Geem and J. C. Williams, “Harmony search and ecological optimization,” International Journal of Energy and Environment, vol. 1, pp. 150–154, 2007.

Z. W. Geem, K. S. Lee, and Y. Park, “Application of harmony search to vehicle routing,” American Journal of Applied Sciences, vol. 2, pp. 1552–1557, 2005.

M. Mahdavi, M. Fesanghary, and E. Damangir, “An improved harmony search algorithm for solving optimization problems,” Applied Mathematics and Computation, vol. 188, no. 2, pp. 1567–1579, 2007.

M. G. H. Omran and M. Mahdavi, “Global-best harmony search,” Applied Mathematics and Computation, vol. 198, no. 2, pp. 643–656, 2008.

C.-M. Wang and Y.-F. Huang, “Self-adaptive harmony search algorithm for optimization,” Expert Systems with Applications, vol. 37, no. 4, pp. 2826–2837, 2010.

Q.-K. Pan, P. N. Suganthan, M. F. Tasgetiren, and J. J. Liang, “A self-adaptive global best harmony search algorithm for continuous optimization problems,” Applied Mathematics and Computation, vol. 216, no. 3, pp. 830–848, 2010.

Q.-K. Pan, P. N. Suganthan, J. J. Liang, and M. F. Tasgetiren, “A local-best harmony search algorithm with dynamic subpopulations,” Engineering Optimization, vol. 42, no. 2, pp. 101–117, 2010.

A. Kaveh and S. Talatahari, “Particle swarm optimizer, ant colony strategy and harmony search scheme hybridized for optimization of truss structures,” Computers and Structures, vol. 87, no. 5-6, pp. 267–283, 2009.

O. M. Alia and R. Mandava, “The variants of the harmony search algorithm: an overview,” Artificial Intelligence Review, vol. 36, no. 1, pp. 49–68, 2011.

Y. Xin-She, “Harmony search as a metaheuristic algorithm,” in Music-Inspired Harmony Search Algorithms: Theory and Applications, Z. W. Geem, Ed., pp. 1–13, Springer, 2009.

S. García, D. Molina, M. Lozano, and F. Herrera, “A study on the use of non-parametric tests for analyzing the evolutionary algorithms' behaviour: a case study on the CEC'2005 Special Session on Real Parameter Optimization,” Journal of Heuristics, vol. 15, no. 6, pp. 617–644, 2009.