Estimation of distribution algorithm (EDA) is an intelligent optimization algorithm based on the probability statistics theory. A fast elitism Gaussian estimation of distribution algorithm (FEGEDA) is proposed in this paper. The Gaussian probability model is used to model the solution distribution. The parameters of Gaussian come from the statistical information of the best individuals by fast learning rule. A fast learning rule is used to enhance the efficiency of the algorithm, and an elitism strategy is used to maintain the convergent performance. The performances of the algorithm are examined based upon several benchmarks. In the simulations, a onedimensional benchmark is used to visualize the optimization process and probability model learning process during the evolution, and several twodimensional and higher dimensional benchmarks are used to testify the performance of FEGEDA. The experimental results indicate the capability of FEGEDA, especially in the higher dimensional problems, and the FEGEDA exhibits a better performance than some other algorithms and EDAs. Finally, FEGEDA is used in PID controller optimization of PMSM and compared with the classicalPID and GA.
Various optimization problems exist in engineering and academic research, which expect to find the best solution. If the problems are conventional or linear, the common mathematical methods will be effective. However, if the problems are too complicated to the common methods, some heuristic algorithms will be considered. Evolutionary algorithms (EAs) are very popular heuristic optimization techniques in the recent years. EAs are general terms of several optimization algorithms that are inspired by the Darwinian theory of natural evolution. It has the capability of solving the complicated optimization problems with nonlinear, high dimension and noncontinuous characteristics. The algorithms search the optimal solution from many possible solutions, and the genetic operators, which simulate the principle of natural genetic evolution, are used to update the individuals. By several iterations, the optimal solution will be obtained, such as the genetic algorithms (GAs) [
In recent years, estimation of distribution algorithms (EDAs) have attracted a lot of attention. It was proposed by Miuhlenbein and Paaß [
The type of probabilistic models used by EDAs and the methods employed to learn them may vary according to the characteristics of the optimization problem. Therefore, different EDAs have been proposed for discrete and continuous problems. In traditional EDAs, the individuals are encoded with binary strings inheritance from EAs. In the populationbased incremental learning (PBIL) algorithm [
In case of realvalued problems, there are some approaches to extend EDAs to other domains, such as mapping other domains to the domain of fixedlength binary strings and then mapping the solution back to the problem’s original domains, or extend or modify the class of probabilistic models to other domains. This first approach might lead to undesirable limitations and errors on realcoded problems. For the second method, the normal pdf is commonly used in continuous EDAs to represent univariate or multivariate distributions. Therefore, some EDAs based on the Gaussian distribution have been designed. In the stochastic hillclimbing with learning by vectors of normal distributions [
EDA is realized by probability estimation and sampling. The probability model is used to estimate the solution distribution, and the probability sampling is used to generate new individuals. In order to improve the performance of standard EDA, we adopt an elitism strategy in FEGEDA. Figure
Flowchart of FEGEDA.
The steps of the FEGEDA are as follows.
Set the population size
Evaluate the
Select
Use the fast learning rule and build the Gaussian normal distribution by the
Make use of sampling technique to generate a new population according to the probability model built in Step
Finally, the iteration is terminated according to the termination criteria. These criteria can be as simple as a fixed number of generations or imply a statistical analysis of the current population to evaluate the stopping condition criteria. If the stopping conditions do not meet, return to Step
The probability model is built according to the distribution of the best solutions in the current population. Therefore, sampling solutions from this model should fall in promising areas with high probability. For some iterations, the solutions should be more likely to be close to the global optimum. The details of the main algorithm are explained in the following.
In the algorithm, little parameters are needed to set except for the population size
In the individuals’ evaluation, it depends on the characteristics of the problem. Conventionally, we should define an objective function
The most important and crucial step of EDAs is the construction of probabilistic model for the solution distribution; to do this step of FEGEDA, Gaussian distribution of individuals is assumed to model and estimate the distribution of promising solutions in every dimension of the problem. Therefore, mean and standard deviation parameters of promising population are required which computed adaptively by maximum likelihood technique.
In order to construct a pdf model of the promising solutions, we should obtain the statistical information of promising solutions. Hence, statistical techniques have been extensively applied to the optimization problems. Fortunately, these parameters can be efficiently computed by the maximumlikelihood estimations [
In the pdf models that assume full independence, every variable is assumed independent of any variable. It must be noted that, in difficult optimization problems, different dependency relations can appear between variables and, hence, considering all of them independent may provide a model that does not represent the problem accurately. However, even if more involved probability models and mixtures of pdfs are defined and used in EDAs, the probability models cannot reflect the problem completely. For system modeling, the dependency relations between variables are very important. Conversely, for optimization problem, the problem decoupled as the combination of some independent variables is expected. Therefore, we specifically focus on the use of independent probability model to construct a fast elitism Gaussian EDA with better performance. That is, the probability distribution
This is very suitable for calculation. Different from the discrete case, the number of parameters to be estimated does not grow exponentially with
The mean and covariance parameters of the normal pdf can be estimated from the selected individuals. Consider
These parameters are always learned in the process of optimization. The iterative learning approaches are used in some literatures [
In this paper, the normal pdf
The probability distribution
The parameters
The probability sampling is used to generate new individuals using the learned probabilistic models. The sampling method depends on the type of probabilistic model and the characteristics of the problem. For normal pdf problem, a conversion is used in order to convert the normal pdf to a standard normal pdf.
Suppose
The normal pdf about
The variable
In the probability models, every variable
Cartogram of sampling data.
Elitism strategy is an effective strategy to ensure that the best individual is selected as the next generation in EAs, because the best individual may include the information of optimal solution. Therefore, elitism can improve the convergence performance of EAs in many cases [
Population operation diagram.
In the simulation, in order to exhibit the performance of FEGEDA, we carry out several different simulations, including onedimensional benchmark, twodimensional benchmarks, and higher dimensional benchmarks. Moreover, we compare the FEGEDA with other EDAs and other kinds of optimization algorithms.
Onedimensional problem is easy for FEGEDA. In order to visualize the information of optimization process and models learning process during the evolution clearly, we carry out a onedimensional benchmark optimization simulation:
The best individuals number
The optimum solutions of each iteration under different conditions.
In Figure
The individuals distribution and probability model of different iterations.
The best selected individuals number is also an important parameter. The convergent speed is faster when the best selected individuals number
Figure
The boxplot of population for different iterations.
In order to testify the optimization capability of FEGEDA further, three twodimensional complex functions are considered:
Function diagrams of
Function
Function
Function
The population size
Comparisons of convergent results.
The advantage of FEGEDA is the capability of higher dimensional problems solution. Some typical benchmarks are considered, including Quadric, Rosenbrock, Ackley, Griewank, Rastrigrin, and Schaffer’s
High dimensional benchmarks.
Quadric 

Rosenbrock 

Ackley 

Griewank 

Rastrigrin 

Schaffer’s 

The algorithm is testified under different
The convergent results under different
We have a comparison of FEGEDA with other EDAs in [
Comparisons of convergent results.
PID is the most used controller in the permanent magnet synchronous motors (PMSM) control. However, PID controller has poor performance in PMSM control due to the inappropriate parameters. During the past decades, great attention has been paid to the stochastic approach, which is potential in dealing with the problem [
The mathematical model of PMSM in a
The magnetic linkage equation can be expressed as follows:
The electromagnetic torque of PMSM in the
According to the motion equation of motor,
Thus, the state equation can be derived from the above equations as follows:
In the VC system of PMSM,
The continuous form of a PID controller, with input
There are two types of discrete PID by discretization of continuous PID. The positiontype discrete PID is described as
Aggregation function is a conventional method, which can convert a multiobjective problem to a singleobjective problem. Consider
In the optimization process, the objective is to evaluate the performance of PIDs. Thus, for PID, the fitness function is written as [
To avoid overshoot, a penalty value is adopted in the fitness function. That is, once overshoot occurs, the value of overshoot is added to the fitness function. Hence, the penalty function is written as
Making use of the aggression function, the fitness function is constructed as follows:
According to state space equation (
MATLAB/Simulink model of PMSM.
The component of PMSM is encapsulated into a module. A speed controller added to the speed closed loop. Figure
The diagram of PMSM control system.
In order to testify the algorithm, GA and traditional PID are selected to compare against FEGEDA.
The system response of PMSM with different PIDs.
We studied the estimation of distribution algorithm in this paper and proposed a fast elitism Gaussian EDA for continuous optimization. Every dimension of individuals is represented by means and standard deviations of Gaussian distribution. These parameters are estimated using maximum likelihood technique by fast learning rule. Then the new population is generated by sampling and elitism strategy. The elitism strategy improves the convergent performance of the algorithm. In the onedimensional test, we exhibit the optimization process and probability models learning process clearly. In the twodimensional and higher dimensional problems, we compare the FEGEDA with danger immune algorithm and other EDAs, and the FEGEDA exhibits a good performance. Although the performance of FEGEDA is promising, further studies are still recommended, for instance, how to increase the diversity of population under fast convergence rate.
The authors declare that there is no conflict of interests regarding the publication of this paper.
This work is supported by the National Natural Science Foundation of China (under Grant 61174044).