Mathematical Problems in Engineering
Volume 2017 (2017), Article ID 6129865, 14 pages
https://doi.org/10.1155/2017/6129865

Research Article

A Study on the Convergence of Family Particle Swarm Optimization

Zhenzhou An,¹ Xiaoyan Wang,¹ and Xinling Shi²

¹Yuxi Normal University, Yuxi 653100, China
²Yunnan University, Yunnan 650091, China

Correspondence should be addressed to Zhenzhou An; anzhenzhou@sina.com

Received 31 August 2016; Revised 18 October 2016; Accepted 30 November 2016; Published 31 January 2017

Academic Editor: Bing Wang

Copyright © 2017 Zhenzhou An et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract.

The sociological concept of family has been introduced in the particle swarm optimization (PSO) and the family PSO (FPSO) has been proposed, in which the particle swarm consisted of different families, each family consisted of different members, and there were different constraint relationships between family members. To further study the sensitivity of FPSO to the control parameters, this paper proposed a special model of FPSO and analyzed the convergence of FPSO theoretically. This model offered a new view to research the particle trajectory and divided the position sequence of particle into the even and odd subsequences. By mathematical analysis, the condition of two subsequences convergence was obtained and the related convergent theories and corollaries were proved. Simulations for benchmark functions showed that the convergence behavior of model and experimental results provided a valuable guideline for selecting control parameters.

1. Introduction

Particle swarm optimization (PSO) is an evolutionary computation algorithm which is motivated by the preying behavior of bird flocking [1]. Due to the simple but efficient characteristics, the PSO has been successfully applied to biomedical image segmentation [2], gene selection and classification [3], and data mining [4], and so forth.

A great deal of theoretical research has been done to study the convergence performance of PSO [5–7]. From these studies it has been concluded that the PSO is sensitive to the choice of control parameters, specifically to the inertia weight and acceleration coefficients. Wrong initialization of these parameters may lead to divergent. To further understand the behavior of particle swarm, some theoretical studies have been done to analyze the trajectory of a single particle in PSO. Ozcan and Mohan [8, 9] concluded that the trajectory of a particle in a simple PSO system was a sinusoidal wave where the initial conditions and parameter choices determined its amplitude and frequency. Van Den Bergh and Engelbrecht [10] developed a model of PSO considering the influence of the inertia weight. Clerk and Kennedy [11] provided a theoretical analysis of particle behavior in which a set of coefficients to control the system’s convergence tendencies were analyzed.

Most of the theoretical studies were based on the simplified PSO models, in which a swarm consisted of one particle of one dimension. The personal best position and the global best position of particles were assumed to be constant throughout the process. Obviously, interactive effects among particles were not taken into account effectively.

To study the interactive effects among particles and enlarge an individual’s cognitive ability, the sociological concept, the so-called family [12, 13] was introduced in the PSO and the family PSO (FPSO) [14, 15] was proposed. When family is considered as a unit, the relationship of family members will be very important. There are different constraint relationships between family members. For example, an equal relationship exists between husband and wife or between siblings; a generational relation exists between parents and children. The different types of relationships among family members mean the different family communication strategies [12, 13]. So the particle swarm consisted of different families, each family consisted of different members and there were different constraint relationships between family members in the FPSO [14, 15].

In this paper, we further analyzed the FPSO theoretically and proposed the special model of FPSO. This model divided the position sequence of particle into the even and odd subsequences. The condition of two subsequences convergence was obtained and the related convergent theories and corollaries were proved. These theories and corollaries demonstrated that the particle trajectory is remarkably different for different parameter sets.

2. Overview of the PSO

PSO is a population-based stochastic optimization technique. In the PSO algorithm, an individual particle is composed of three vectors: its position , the best position found by itself , and its velocity . Particles are originally initialized in a uniform random manner throughout the search space. Then, their positions are changed according to their own experience and that of the entire swarm.

The velocity and position are defined by the following rules:

where , ; and are called acceleration coefficients; and are uniformly distributed pseudorandom numbers in the range of 0-1; is the personal best position, and is the best position found by the swarm at the th iteration. The constriction factor is defined by Clerc and Kennedy [11].

3. The Performance Analysis of the FPSO

3.1. Description of the FPSO

In the FPSO [14, 15], the particle swarm consists of different families. Every family has more than one member. Every member in the family provides the information got by the previous experience to other family members.

The velocity and position of particle are updated by the following rules (suppose particle belongs to the th family):

where is the best position found by the th family and is the best position found by all families at the th iteration.

The particle position update equation can by derived by the following transform:

  Substituting into (3),

  Substituting into (4),

By substitution, we obtain that the position of the th particle is updated by the following second-order nonhomogeneous linear differential equation:

Let

we have

3.2. The Special Model of FPSO

The constraint relationship among the parameters , , and can be obtained through studying the coefficients of and in (8). To analyze the parameters of FPSO, there are two kinds of noticeable parameter settings: one is and another is .

When , the previous velocity will not influence the new velocity and the memory of the previous flight direction will be erased. Equation (8) can be simplified as shown below:

When , , , (8) can be simplified as shown below:

In this section, will be particularly analyzed.

From (4), can be obtained:

By recurrence, (10) can be derived:

So the positions of the th particle form two subsequences: and . One is related to and at odd iteration steps and another is affected to and at even iteration steps.

3.3. The Convergent Property of the Special FPSO Model

Theorem 1. If , for all positive integrate ,

Proof. (1) If , we trivially have

(2) Assume that the equation is true for : that is,

Then we need to show that the equation continues to hold for .
From (10), can be obtained:

From (10), can be obtained:

So, if , (13) are obviously true.
Therefore, the statement is true for all positive integers .

Theorem 2. If and , the sequences and converge, respectively.

Proof. When ,

First, we prove the limit exists.
Let :

When ,

So that, , when , , according to the Cauchy Convergence Theorem, converges. That is, converges and the limit exists.
Second, we prove the limit exists.

Because converges and the limit exists, the sequence converges and the limit exists.
Likewise, the convergence of may be proved and the limit exists. So that the sequence converges and the limit exists, too.

Corollary 3. If particles and belong to the same family, then

Proof. If particles and belong to the th family, then .
Thus,

When , ,

This means if particles and belong to the same family, the even and odd sequences of and will converge to the same point, respectively.

Let

Corollary 4. If particle belongs to the th family and particle belongs to the th family, then

Proof. One has

When , .
So

This means if particles and belong to the different families, the distances of the even and odd sequences of and will converge to the invariable values, respectively.

Corollary 5. If and , then

Proof. One has

This means if and , the even and odd sequences of will converge to the same point.

Corollary 6. If the family best position and the global best position are assumed to be constant throughout the process, that is, , , then

In particular,

Proof. One has

When ,

Likewise,

This means if the family best position and the global best position are assumed to be constant throughout the process, the even and odd sequences of will converge to the weighted average of the family best position and the global best position; if , they will converge to the global best position; if , they will converge to zero.

Corollary 7. When , if , the sequences and exponentially decay to zero without oscillating; if , the sequences and oscillate with the amplitude gradually decreasing to zero.

Proof. When , , .
(1) ,

There are four cases:

(i)If , then .(ii)If , then .(iii)If , then .(iv)If , then .

So , have the same sign as , , respectively. When , gradually decreases with the increase of variable . So the sequences and exponentially decay to zero without oscillating.
(2) ,

There are four cases:

(i)If , then , .(ii)If , then , .(iii)If , then , .(iv)If , then , .

Whether the sign of or , and have the opposite sign by the above cases. and have the same feature. When , with the increase of variable , oscillates with the amplitude gradually decreasing. So the sequences and oscillate with the amplitude gradually decreasing to zero.

4. Experiments

In order to analyze the parameters, extensively adopted benchmark functions were used in the experiments, as listed in Table 1. The functions were simple unimodal problems; were highly complex multimodal problems with many local minima. A detailed description of these functions could be found in [16].

Table 1: Mathematical representation of test functions.

Function
Mathematical representation
Domain

Sphere

Schwefel’s Problem  1.2

Generalized Rastrigin

Ackley

Generalized Griewank

4.1. Random Examples of the Even and Odd Properties of FPSO

In order to facilitate the comparison, the population size was four particles that included two families and every family had two particles in the experiments. Because different benchmark functions had different domains, the positions and velocities of particles were initialized in the experiments, respectively. In our experiments, and were generated randomly. The choices of parameters satisfied the following equation: . Simulations of particle trajectory for these parameters were given in Figures 1–7.

(a) The overall trajectory

(b) The even and odd trajectories

Figure 1: , for Sphere.

(a) The overall trajectory

(b) The even and odd trajectories

Figure 2: , for Schwefel’s Problem 1.2.

(a) The overall trajectory

(b) The even and odd trajectories

Figure 3: , for Generalized Rastrigin.

(a) The overall trajectory

(b) The even and odd trajectories

Figure 4: , for Ackley.

(a) The overall trajectory

(b) The even and odd trajectories

Figure 5: , for Generalized Griewank.

(a) The overall trajectory

(b) The even and odd trajectories

Figure 6: , for Schwefel’s Problem 1.2.

(a) The overall trajectory

(b) The even and odd trajectories

Figure 7: , for Generalized Griewank.

In Figures 1–7, part (a) represented the overall trajectories of different particles in different families; part (b) represented the even and odd trajectories of them. The parameters satisfied the following equation: . According to Theorem 1, the overall trajectories could be divided into the sequences and .

In Figures 1–5, the parameters satisfied the following inequality: . According to Theorem 2, the sequences and converged, respectively. In Figures 6 and 7, the relationship of the parameters was . According to Theorem 2, the sequences and did not converge, respectively.

4.2. Special Examples of the Even and Odd Properties of FPSO

According to Corollary 7, if , the sequences and exponentially decay to zero without oscillating; if , the sequences and oscillate with the amplitude gradually decreasing to zero. Two special choices of parameters were used in these examples. One of them was ; another was . Each choice had four cases to and , which included and , and , and , and and . In this section, Schwefel’s Problem 1.2 was mainly shown and its initialized values were shown in Table 2. Simulations of particle trajectory for these parameters were given in Figures 8 and 9.

Table 2: Initial parameters for Schwefel’s Problem  1.2.

Family
Particle
Initial parameters

Family 1
Particle 1

Particle 2

Family 2
Particle 3

Particle 4

(a) The overall trajectory

(b) The even and odd trajectories

Figure 8: , for Schwefel’s Problem 1.2.

(a) The overall trajectory

(b) The even and odd trajectories

Figure 9: , for Schwefel’s Problem 1.2.

In Figures 8 and 9, part (a) represented the overall trajectories of different particles in different families; part (b) represented the even and odd trajectories of them.

In Figures 8 and 9, and , respectively, so . According to Theorem 2, the sequences and converged, respectively.

4.3. Examples of the Convergent Property of FPSO

The following examples represented how the convergent property of FPSO was affected by the different choices of the algorithm parameters , , and . Simulations of particle trajectory according to related parameters were given in Figures 10–13.

(a)

(b) and were changing with the iteration times

Figure 10: , for Schwefel’s Problem 1.2.

(a)

(b) and were changing with the iteration times

Figure 11: , for Schwefel’s Problem 1.2.

(a)

(b) and were changing with the iteration times

Figure 12: , for Schwefel’s Problem 1.2.

(a)

(b) and were changing with the iteration times

Figure 13: , for Schwefel’s Problem 1.2.

The algorithm parameters were selected and satisfied in our experiments; according to Theorem 2, the sequences and converge, respectively.

In Figures 10–13, part (a) represented the family best position and the global best position that were the same and equaled zero; part (b) represented the family best position and the global best position that were variable with the iteration times.

In Figures 10 and 11, and , respectively, so ; according to Corollary 7, the sequences and exponentially decayed to zero without oscillating. When , the results revealed that and were bigger and the convergent speed was faster.

In Figures 12 and 13, and , respectively, so ; according to Corollary 7, the sequences and oscillated with the amplitude gradually decreasing to zero. When , the results revealed that and were bigger and the convergent speed was slower.

5. Conclusions

To study the sensitivity of FPSO to the choice of control parameters, this paper proposed a special model of FPSO. This model divided the position sequence of particle into the even and odd subsequences: and .

By analysis theoretically, when the algorithm parameters were selected and satisfied and , the sequences and converged, respectively; the related convergent theories and corollaries were proved.

Simulations for benchmark functions demonstrated that the particle trajectory could be divided into the even and odd subtrajectories. Experimental results demonstrated that the particle trajectory of FPSO was remarkably affected by the different parameter choices. These results provided a valuable guideline for selecting control parameters.

Future research is needed to find better parameter sets in the convergence domain, the effect of the randomness and the interaction among particles in a family, and so forth.

Competing Interests

The authors declare that they have no competing interests.

Acknowledgments

This work was supported by the Scientific Research Fund of Yunnan Provincial Education Department (2015Z178) and Natural Science Foundation of China (61563055).

References

J. Kennedy and R. Eberhart, “Particle swarm optimization,” in Proceedings of the IEEE International Conference on Neural Networks, pp. 1942–1948, Piscataway, NJ, USA, December 1995.

S. Lahmiri and M. Boukadoum, “Combined partial differential equation filtering and particle swarm optimization for noisy biomedical image segmentation,” in Proceedings of the 7th IEEE Latin American Symposium on Circuits and Systems (LASCAS '16), pp. 363–366, Florianopolis, Brazil, March 2016.

D. Ramyachitra, M. Sofia, and P. Manikandan, “Interval-value based particle swarm optimization algorithm for cancer-type specific gene selection and sample classification,” Genomics Data, vol. 5, pp. 46–50, 2015.

R. J. Kuo, C. M. Chao, and Y. T. Chiu, “Application of particle swarm optimization to association rule mining,” Applied Soft Computing, vol. 11, no. 1, pp. 326–336, 2011.

T. Beielstein, K. E. Parsopoulos, and M. N. Vrahatis, “Tuning pso parameters through sensitivity analysis,” Technical Report, Reihe Computational Intelligence CI 124/02, Department of Computer Science, University of Dortmund, Dortmund, Germany, 2002.

Y. Shi and R. C. Eberhart, “A modified particle swarm optimizer,” in Proceedings of the IEEE Congress on Evolutionary Computation, pp. 69–73, Piscataway, NJ, USA, 1998.

I. C. Trelea, “The particle swarm optimization algorithm: convergence analysis and parameter selection,” Information Processing Letters, vol. 85, no. 6, pp. 317–325, 2003.

E. Ozcan and C. K. Mohan, “Analysis of a simple paticle swarm optimization system,” in Intelligent Engineering Systems through Artificial Neural Networks, pp. 253–258, 1998.

E. Ozcan and C. K. Mohan, “Particle swarm optimization: Srufing the waves in,” in Proceedings of the 1999 Congress on Evolutionary Computation (CEC '99), Washington, DC, USA, 1999.

F. Van Den Bergh and A. P. Engelbrecht, “A study of particle swarm optimization particle trajectories,” Information Sciences, vol. 176, no. 8, pp. 937–971, 2006.

M. Clerc and J. Kennedy, “The particle swarm: explosion, stability, and convergence in a multidimensional complex space,” IEEE Transactions on Evolutionary Computation, vol. 6, no. 1, pp. 58–73, 2002.

E. W. Burgess and H. J. Locke, The Family: From Institution to Companionship, American Book Company, New York, NY, USA, 1945.

X. T. Fei, Shengyu zhidu (The Regime of Childbirth), Tianjin People's Publishing House, Tianjin, China, 1981.

Z. Z. An, X. L. Shi, and J. H. Zhang, “Family particle swarm optimization,” in Proceedings of the 6th International Conference on Wireless Communications, Networking and Mobile Computing (WiCOM '10), IEEE, Chengdu, China, September 2010.

Z. An, X. Shi, and J. Zhang, “A study on the internal structure of family particle swarm optimization,” in Proceedings of the 4th International Conference on Genetic and Evolutionary Computing (ICGEC '10), pp. 27–30, IEEE Computer Society, December 2010.

R. Mendes, Population Topologies and Their Influence in Particle Swarm Performance, University of Minho, 2004.