A Modified Bat Algorithm with Conjugate Gradient Method for Global Optimization

Metaheuristic algorithms are used to solve many optimization problems. Fireﬂy algorithm, particle swarm improvement, harmonic search, and bat algorithm are used as search algorithms to ﬁnd the optimal solution to the problem ﬁeld. In this paper, we have investigated and analyzed a new scaled conjugate gradient algorithm and its implementation, based on the exact Wolfe line search conditions and the restart Powell criterion. The new spectral conjugate gradient algorithm is a modiﬁcation of the Birgin and Mart´ınez method, a manner to overcome the lack of positive deﬁniteness of the matrix deﬁning the search direction. The preliminary computational results for a set of 30 unconstrained optimization test problems show that this new spectral conjugate gradient outperforms a standard conjugate gradient in this ﬁeld and we have applied the newly proposed spectral conjugate gradient algorithm in bat algorithm to reach the lowest possible goal of bat algorithm. The newly proposed approach, namely, the directional bat algorithm (CG-BAT), has been then tested using several standard and nonstandard benchmarks from the CEC’2005 benchmark suite with ﬁve other algorithms and has been then tested using nonparametric statistical tests and the statistical test results show the superiority of the directional bat algorithm, and also we have adopted the performance proﬁles given by Dolan and More which show the superiority of the new algorithm (CG-BAT).


Introduction
In 2010, Yang proposed a new optimization algorithm, namely, bat algorithm (BA), based on swarm intelligence and the inspiration from observing the bats. Although the original BA presents superior results in the experiments than PSO, we notice that the performance and the accuracy of the original BA still have the capacity to present better. e algorithm exploits the so-called echolocation of bats.
Bats use sonar echoes to detect and avoid obstacles. It is generally known that sound pulses are transformed to frequency which reflects from obstacle. Bats can use time delay from emission to reflection and use it for navigation. ey typically emit short loud, sound impulses. e pulse rate is usually defined as 10 to 20 times per second. After hitting and reflecting, the bats transform their own pulse into useful information to gauge how far away the prey is.
e bats are using wavelengths that vary from the range 0.7 to 17 mm or inbound frequencies of 20-500 kHz. To implement the algorithm, the pulse frequency and rate have to be defined. e pulse rate can be simply determined in the range from 0 to 1, where 0 means that there is no emission and 1 means that bats are emitting maximum [1]. e bat-inspired algorithm is a recent swarm-based intelligent system which mimics the echolocation system of microbats. In the bat-inspired algorithm, the bats randomly fly around the best bat locations found during the search so as to improve their hunting of prey. In practice, one bat location from a set of best bat locations is selected. ereafter, that best bat location is used by local search with a random walk strategy to inform other bats about the prey location. is selection mechanism can be improved using other natural selection mechanisms adopted from other advanced algorithms such as genetic algorithm. erefore, six selection mechanisms are studied to choose the best bat location: global-best, tournament, proportional, linear rank, exponential rank, and random. Consequently, six versions of the bat-inspired algorithm are proposed and studied which are global-best bat-inspired algorithm (GBA), tournament bat-inspired algorithm (TBA), proportional bat-inspired algorithm (PBA), linear rank batinspired algorithm (LBA), exponential rank bat-inspired algorithm (EBA), and random bat-inspired algorithm (RBA). Using two sets of global optimization functions, the bat-inspired versions are evaluated and the sensitivity analyses of each version to its parameters studied [2]. A success of an algorithm always depends on well balanced of these components. e aim of this study is to improve the performance of the standard bat algorithm by increasing its exploration and exploitation abilities along the main line of the BA. In this paper, two improvement strategies are presented. e first improvement strategy is the development of a spectral conjugate gradient technique, which can be used to guide the research process, and the second improvement strategy is to improve the bat algorithm using the conjugate gradient method to arrive at the best solution for the current iteration, which can be used to enhance the ability to local search. e newly proposed optimally directional bat algorithm (CG-BAT) will be tested on several benchmark problems chosen from the well-known CEC'2005 benchmark set and compared with several other swarm and evolutionary algorithms. erefore, this study is organized as follows. A new scalar in a spectral conjugate gradient is described in Section 2. A global convergence is described in Section 3, the standard bat algorithm is presented in Section 4. en, the enhanced bat algorithm is presented in Section 5. Finally, the results of the numerical experiments are presented in Section 6, followed by the conclusions in Section 7. Too much exploration but too little exploitation may cause difficulties that algorithm converges towards optimal solutions. e conjugate gradient technique could be a helpful procedure to search out the minimum value of any nonlinear function to find optimal solutions: where f : R n ⟶ R is a real-valued function. e numerical formula is given by where α k is a step length to be computed by a line search procedure [3]. e search direction d T k � − g k is outlined as follows: and g k � ∇f(x k ) is gradient and β k is a parameter of conjugacy condition. Some famed formulas of this parameter are outlined as follows: where y k � g k+1 − g k , which are referred to as Hestenes-Stiefel (HS) [4], Polak-Ribière (PR) [5], Fletcher-Reeves (FR) [6], and Dai-Yuan (DY) [7] severally. Many authors have studied the convergence of the on top of formulas for years [8][9][10][11].
To prove the convergence analysis of the conjugate gradient technique, the following weak Wolfe conditions are used: Used the strong Wolfe conditions consist of (5) and e constants are within the period 0 < δ < σ < 1, and additional details are found in [3]. Well, the sufficient descent property is defined as follows: where ‖·‖ denotes the Euclidean norm, provided that c is any positive constant [7].

A New Scalar in Spectral CG Method (ξ k+1 )
Birgin and Martínez (SS) [12] instructed a spectral conjugate gradient technique outlined by where s k � α k d k . e parameter β k has the following form: 2 International Journal of Mathematics and Mathematical Sciences where the spectral ξ k+1 in [13] is determined by using the following equation: In this section, we will derive a new spectral CG method as follows: e matrix H k+1 is asymmetrical and positive definite and the scalar ρ ⌣ k+1 is defined by Al-Bayati and Salah [14] as ρ ⌣ k+1 � (y T k s k /g T k H k g k ). By equating (9) and (12), we get Multiplying both sides of (13) by y T k , we get Since d k � − H k g k and y T k H k+1 � s T k , we get y T k s k g T k d k Since s k � α k d k , If we use exact line search, then the new scalar ξ k+1 is equal to one. e new direction is defined by the following equation: Proof. Under some algebraic operations, the direction of (17) can be written as follows: Now, multiplying both sides of (18) by (g k+1 /‖g k+1 ‖ 2 ), then we get Since s T k g k+1 ≤ s T k y k and y T k g k+1 ≤ ‖y k ‖·‖g k+1 ‖, we get Since International Journal of Mathematics and Mathematical Sciences Let υ � (‖y k ‖/‖g k+1 ‖) (where υ is the positive constant), then □

Global Convergence
In this section, the subsequent assumption is usually used in proving the global conjugate gradient methods.
Assumption 1 (see [15]) In neighborhood N of S, the function f is continuously differentiable and its gradient is Lipschitz continuous, i.e., there exists a constant L > 0 such that Below the assumptions (i) and (ii) on f, we can deduce that there exists c > 0 such as the following equations: Lemma 1 (see [16]). Assume that Assumption 1 holds and suppose that, for any conjugate gradient method, d k+1 is a descent direction and the step size α k satisfies conditions (5) and (7) if en Theorem 2. Suppose that Assumption 1 holds, and the direction d k+1 defined by (17) is descent and α k is computed using (5) and (7), then Proof. By using some algebraic operations of (17) and taking the absolute value, we get Since s T k g k+1 ≤ s T k y k and y T k g k+1 ≤ ‖y k ‖ · ‖g k+1 ‖, we get that is, Lim k⟶∞ ‖g k ‖ � 0, the proof is complete.

Standard Bat Algorithm
e bat algorithm proposed by Yang [17] is an intelligent optimization algorithm inspired by the echolocation behavior of bats. When flying and hunting, bats emit some short, ultrasonic pulses to the environment and list to their echoes. Studies show that the information from the echoes will enable bats to build a precise image of their surroundings and determine precisely the distance, shapes, and prey's location.
e capability of such echolocation of microbats is fascinating, as these bats can find their prey and discriminate against different types of insects even in complete darkness [17]. e earlier studies showed that BA can solve unconstrained optimization problems with much more efficiency and robustness compared to GA and PSO [18,19]. e used idealized rules in bat algorithm are as follows: (a) All bats use echolocation to sense distance and the location of a bat x i is encoded as a solution to an optimization problem under consideration. (b) Bats fly randomly with velocity v i at position x i with a varying frequency (from a minimum f min to a maximum frequency f max ) or a varying wavelength λ and loudness A to search for prey. ey can automatically adjust the wavelengths (or frequencies) of their emitted pulses and the rate of pulse emission r depending on the proximity of the target. (c) Loudness varies from a large positive value A 0 to a minimum constant value A min [17].
For each bat (i), its position (x i ) and velocity (v i ) in a ddimensional search space should be defined. x i and v i should be subsequently updated during the iterations. e rules for updating the position and velocities of a virtual bat (i) are given as in [17]: where rand ∈ [0, 1] is a random vector drawn from a uniform distribution.
Here, x * is the current global best location (solution) which is located after comparing all solutions among all the n bats. A new solution for each bat is generated locally using random walk given by where ε ∈ [− 1, 1] is a random number, while 〈A t+1 k 〉 is the average loudness of all the bats at this time step. e loudness A k and the rate of pulses emission r k are updated as the iterations proceed. e loudness decreases and the pulse rate increases as the bat gets closer to its prey. e equation for updating the loudness and the pulse rate is given by where 0 < ϖ < 1 and η > 0 are constants. As t ⟶ ∞, we have A t k ⟶ 0 and r t k ⟶ r 0 k . e initial loudness A 0 can typically be A 0 ∈ [1, 2], while the initial emission rate r 0 ∈ [0, 1]. e basic steps of the standard bat algorithm are summarized in the pseudocode as shown in Algorithm 1.

Enhanced Bat Algorithm
is paper attempts to improve the bat algorithm from a different perspective from the previous improvements by hybridizing the bat method using optimization methods, by using the optimal size of the cubic step and the optimal search direction for the synchronous gradient feature of the optimal search direction for echo detection. First, local movements can be improved by controlling the optimum step sizes, while the second bat movement should be directed by other bats and the best local moves toward optimal movement. More specifically, two different adjustments will be made to improve the efficiency of the bat algorithm.

e First Modification (Optimal
Step Size). e first modification concerns local search mechanisms: in standard bats, they are allowed to move from their current locations to new random locations using local random walk. In modified bats, they are allowed to switch from their current locations to new locations optimally using local optimal walking, as we adjust this step to the optimal size using one of the optimization methods called optimal step size (α k ) when the step length is calculated by performing a line search [1].

e Second Modification Using New Spectral Conjugate
Gradient Method. A bat emits two pulses in two different directions, one to the direction of the bat with the best position (the best solution is steepest descent) and the other to the direction of the new conjugate gradient bat. From the echoes, the bat can know if the food exists around these two bats or not. e best position is determined by the objective fitness, while, around the optimally selected bat, it depends on its fitness value. If it has a better fitness value as the actual bat, then the food is considered to exist; otherwise, there is not a food source in the neighborhood. If the food is confirmed to exist around the two bats (Choice 1), the current bat moves to a direction at the surrounding neighborhood of the two bats where the food is supposed to be plenty. If not (Choice 2), it moves toward the best bat. e mathematical formulas of the bats' movements are thus given by e directions of the movement generated by equation (17) are directed towards the bat with the best position. is mechanism allows the BA to exploit more around the best position; however, if the best bat is not near the global optimality, there is a risk that the solutions generated by such moves could be trapped in local optima.
e new proposed movement in equation (17) has the ability to diversify the movement directions which can enhance the exploration capability, especially at the different repetition stages especially at the initial stages of iterations, and can thus avoid premature convergence. Furthermore, when it approaches the end of the iteration process, the bats tend to get around the best bats with stronger exploitability which in turn can reduce the distances between them and thus enhance the speed of convergence which gives stability to the algorithm.
e new algorithm CG-BAT is illustrated by presenting the algorithm and flowchart as follows.

CG-BAT Algorithm.
In this section, we develop the movement of bat algorithm to reach the goal by using the new direction which is defined in equation (17).
Define pulse frequency f k at x k . (4) Initialize pulse rates r k and the loudness A k . (5) While (t ≤ t max ). (6) Adjust frequency equation (1) International Journal of Mathematics and Mathematical Sciences 5

CG-BAT Flowchart.
In this section, we describe the movement of CG-BAT algorithm to reach the goal by using the flowchart of new search direction which is defined in equation (17) (Figure 1).

Experimental Results and Comparisons
To prove the efficiency performance of all newly proposed algorithms, two comparison experiments have been conducted. e first is a comparison between the new spectral conjugate gradient and the standard algorithms in this field, and the second experiment is a comparison between the new bat algorithm (CG-BAT) and cuckoo search, firefly search, and practical swarm.

Experimental Results and Comparisons in New CG.
In this section, we have reported some numerical experiments that are performed on a set of 30 unconstrained optimization test problems to analyze the efficiency of ξ k+1 . Detail of these test problems, with their given initial points. e termination criterion used in our experiments is ‖g k ‖ ≤ 10 − 6 , where δ � 0.001 and σ � 0.1. In our comparisons below, we employ the following algorithms:  Table 1 shows the numerical computations of these newly proposed CG algorithms against other well-known CG-algorithms to check their performance and we have used the following well-known measures or tools used normally for this type of comparison of CG algorithms: NOI � the total number of iterations NOF � the total number of function evaluation TIME � the total CPU time required for the processor to execute the CG algorithm and reach the minimum value of the required function minimization To evaluate the modified conjugate gradient technique, this technique is analyzed and tested in some numerical tests (see [20]) and to demonstrate the performance of those methods, we applied Dolan and Moré [21], a new tool to analyze the efficiency of algorithms. ey introduced the notion of a performance profile as means to evaluate and compare the performance of the set of (1) define objective function f(x) � (x 1 , x 2 , · · · , x d ) T (2) initialize the bat population x k and for k � 1, · · ·, n (3) Define pulse frequency f k at x k . (4) Initialize pulse rates r k and the loudness A k . Requiring a baseline for comparisons, they compared the performance on problem p by solver s with the best performance by any solver on this problem, based on the performance ratio: Suppose that a parameter r M ≥ r P for all p, s is chosen, and r p,s � r M if and only if solver s does not solve problem p (Figure 2). Figure 3 shows the Dolan-More performance profile for these methods, which are subject to the frequency of a suitable performance compared to the basic methods. Figure 4 shows us through the Dolan-More performance profile for these methods, which are measured by the CPU time, which makes us deduce from the three forms presented. e new method is very suitable for solving issues of many dimensions.

Experimental Results and Comparisons in CG-BAT.
To validate the performance of the proposed optimally directional bat algorithm, we have carried out various numerical experiments that have been then tested using several standard and nonstandard benchmarks from the CEC'2005 benchmark suite, which can be summarized as two comparison experiments. e first one is a comparison between the new directional bat algorithm and the standard algorithms including the bat algorithm on the classical benchmark functions, cuckoo search, firefly search, and practical swarm and the second one is a comparison has been performed against some advanced optimization algorithms  International Journal of Mathematics and Mathematical Sciences such as Dolan and Moré [19], a new tool to analyze the efficiency of algorithms. Tables 2-4. We have been used to verify the performance of the new bat algorithm (CG-BAT), compared with that of standard BA, FA, CS, and PSO. e description and the setting parameters of these algorithms are as follows:
(4) CS: the cuckoo search via Lèvy flights described in [22] is considered with the probability of the discovery of alien egg spa � 0.25. (5) PSO: a classical particle swarm optimization [23,24] model has been considered. e parameter settings are c 1 � 1.5 and c 2 � 1.2 and the inertia coefficient w is a monotonically decreasing function from 0.9 to 0.4.
For a fair comparison, the common parameters are considered the same. e population size was set to N � 50, and the number of function evaluations is the same as 15000, without counting the initial evaluations, though all algorithms were initialized randomly in the similar manner. erefore, we set t max � 500 except for CS. Due to the fact    International Journal of Mathematics and Mathematical Sciences 9    e global minimum obtained after each trial was recorded for further statistical analysis. Subsequently, the mean value of the global minimum, the standard deviation (SD), the best solution, the median, and the worst solution values have been computed and are presented in Tables 2-4. From the results presented in Tables 2-4, the new directional bat algorithm achieved better results for 20 functions (F1, F2, F3, F4, F8, F10, F12,  F13, F14, F15, F16, F18, F19, F20, F22, F23, F24, F25, F28, and F29), while the BA obtained better results for 3 functions (F7, F26, and F27). e FA has better scores for 2 functions (F5 and F17). CS obtained best results for F9, F21, and F30 and PSO for F11. We can show that the following monotonically decreasing function is more suitable and gives stability to the algorithm.

6.2.3.
e Second Experiment (Nonparametric Statistical Tests). In this section, to evaluate CG-BAT performance, nonparametric statistical tests were carried out. We performed Friedman's test and pairwise comparisons. Table 5 shows the descriptive statistics for the five algorithms, which gives the number of values studied, the mean and the standard deviation, and the highest value of the values for each method and the lowest value for it. Note that CG-BAT has the least arithmetic mean of 72929.28687 and the standard deviation of 372879.0388, which is lower than the rest of the algorithms, with the lowest and greatest value for the method. Table 6 presents the Friedman rank test. For this test, an algorithm is considered better if it has a low rank. From the results, CG-BAT has the lowest rank for the two tests, which means that it is the best performing algorithm from the comparison. In addition, the last two rows present the test statistic and p value. e statistic is distributed according to the chi-square distribution with 4 degrees of freedom. e lower p value of the different tests suggests the existence of significant differences among the considered algorithms at α � 0.01 level of significance.
To highlight the differences between CG-BAT with each of the other algorithms, Table 7 presents the pairwise comparison results using the Friedman test. e control method is CG-BAT. e analysis of the Friedman rank test shows significant differences between CG-BAT and four algorithms (BA, FA, CS, and PSO) according to the p values of chi-square statistic which all are less than α � 0.05 level of significance, and since CG-BAT algorithm for all pairwise comparisons has a minimum rank (1.28 with BA, 1.24 with FA, 1.24 with CS, and 1.10 with PSO), the results reveal that CG-BAT algorithm is significantly superior to BA, FA, CS, and PSO algorithms.

e ird Experiment (Convergence Curve Analysis).
e convergence curve is an important indicator for the performance of the algorithm, through which we can see the convergence speed and the ability of the new algorithm optimum. In order to evaluate the modified CG-BAT, this technique is analyzed and tested in some numerical tests, and to illustrate the performance of these methods, we applied Dolan and Moré [21] to analyze the efficiency of the algorithm. Performance profiles based on mean performance, standard deviation (SD), and best solution are shown in Figures 5-7.

Conclusions
In this study, we have submitted new spectral CG methods. A crucial property of proposed CG methods is that it secures sufficient descent directions. Under mild conditions, we have demonstrated that the new algorithms are globally convergent for each uniformly convex and general functions using the strong Wolfe line search conditions. e preliminary numerical results show that new algorithms perform very well and also an improved version of the standard bat algorithm, called the new directional bat algorithm (CG-BAT), has been proposed and presented. Two modifications have been embedded to the BA to increase its exploitation and exploration capabilities and consequently have significantly enhanced the BA performance.
ree sets of experiments have been carried out to prove the superiority of the proposed CG-BA). e performance is compared by using thirty test functions, under seven optimization algorithms (SS, HS, New, CG-BAT, BA, CS, FA, and PSO). e comparison results show that the enhanced algorithms (New and CG-BAT) are better than the original algorithms and have relatively stable performance in both the optimization ability and the convergence speed. Performance profile: 2.133200e + 00 Data Availability e data used to support the findings of this study are available from the corresponding author upon request.

Conflicts of Interest
e authors declare that there are no conflicts of interest regarding the publication of this paper.