Particle swarm optimization (PSO) is a heuristic global optimization method, proposed originally by Kennedy and Eberhart in 1995. It is now one of the most commonly used optimization techniques. This survey presented a comprehensive investigation of PSO. On one hand, we provided advances with PSO, including its modifications (including quantumbehaved PSO, barebones PSO, chaotic PSO, and fuzzy PSO), population topology (as fully connected, von Neumann, ring, star, random, etc.), hybridization (with genetic algorithm, simulated annealing, Tabu search, artificial immune system, ant colony algorithm, artificial bee colony, differential evolution, harmonic search, and biogeographybased optimization), extensions (to multiobjective, constrained, discrete, and binary optimization), theoretical analysis (parameter selection and tuning, and convergence analysis), and parallel implementation (in multicore, multiprocessor, GPU, and cloud computing forms). On the other hand, we offered a survey on applications of PSO to the following eight fields: electrical and electronic engineering, automation control systems, communication theory, operations research, mechanical engineering, fuel and energy, medicine, chemistry, and biology. It is hoped that this survey would be beneficial for the researchers studying PSO algorithms.
Artificial intelligence (AI) is the intelligence exhibited by machines. It is defined as “the study and design of intelligent agents” [
Swarm intelligence (SI) is a part of EC. It researches the collective behavior of decentralized, selforganized systems, natural or artificial. Typical SI systems consist of a population of simple agents or boids interacting locally with one another and with their environment. The inspiration often comes from nature, especially biological systems [
The agents in a SI system follow very simple rules. There is no centralized control structure dictating how individual agents should behave. The agents’ real behaviors are local, and to a certain degree random; however, interactions between such agents lead to the emergence of “intelligent” global behavior, which is unknown to the individual agents. Wellknown examples of SI include ant colonies, bird flocking, animal herding, bacterial growth, and fish schooling.
Dorigo [
The distribution of all publications and publication per year w.r.t. SIbased algorithms is presented in Figures
Number of all publications w.r.t. SIbased algorithms.
Publication per year (2010–2014) w.r.t. SIbased algorithms.
Several public websites related to PSO (Table
Public websites of PSO.
Website name  URL 

Particle swarm optimization 

Particle swarm optimization 

Particle swarm central 

PSO toolbox 

PSO visualization 

Particle swarm optimization toolbox 

This work first checked the coherency of PSO with principles required by SI. Second, we reviewed the studies on advances of PSO. Third, various applications of PSO is given. Finally, we conclude the paper by summarizing the improvements and analyzing potential research directions. This survey was carried out mainly by examining the “Web of Science Core Collection” database. In addition, IEEE Explorer and Google Scholar were also used.
Selforganization is a key feature of SI system. It is a process where global order or coordination arises out of the local interactions between the components of an initially disordered system. This process is spontaneous; that is, it is not controlled by any agent inside or outside of the system. Bonabeau et al. [
Strong dynamical nonlinearity (often involving positive and negative feedback): positive feedback helps promote the creation of convenient structures, while negative feedback counterbalances positive feedback and helps to stabilize the collective pattern.
Balance of exploitation and exploration: SI identifies a suitable balance to provide a valuable mean artificial creativity approach.
Multiple interactions: agents in the swarm use information coming from neighbor agents so that the information spreads throughout the network.
In addition, Millonas [
Five basic principles of SI.
Principle  Definition 

Proximity principle  The swarm should be able to do simple space and time computations 
Quality principle  The swarm should be able to respond to quality factors in the environment 
Diverse response principle  The swarm should not commit its activities along excessively narrow channels 
Stability principle  The swarm should not change its mode of behavior every time the environment changes 
Adaptability principle  The swarm should be able to change its behavior mode when it is worth the computational price 
PSO performs searching via a swarm of particles that updates from iteration to iteration. To seek the optimal solution, each particle moves in the direction to its previously best (
It is common to set an upper bound for the velocity parameter. “Velocity clamping” [
The first part of formula (
Let
For each particle
(a) Initialize the particle’s position with a uniformly distribution as
(b) Initialize
(c) Initialize
(d) Initialize velocity:
For each particle
(a) Pick random numbers:
(b) Update particle’s velocity. See formula (
(c) Update particle’s position. See formula (
(d) If
(i) Update the best known position of particle
(ii) If
(e)
In this review, we center in reporting the advances on PSO in the form of formal publications. We divide advances into following six aspects:
modifications of PSO, including quantumbehaved PSO, barebones PSO, chaotic PSO, fuzzy PSO, PSOTVAC, oppositionbased PSO, topology, and other slight modifications,
hybridization of PSO with other metaheuristic methods, including genetic algorithm (GA), artificial immune system (AIS), Tabu search (TS), ACO, simulated annealing (SA), ABC, DE, biogeographybased optimization (BBO), and harmonic search (HS),
extensions of PSO to other optimization fields, including multiobjective, constrained, discrete, and binary optimization,
theoretical analysis of PSO, parameter selection and convergence analysis.
parallel implementation of PSO, including multicore, GPU computing, and cloud computing.
Some researchers proposed quantumbehaved PSO (QPSO), which was motivated by concepts from quantum mechanics. For example, Jau et al. [
The barebones PSO (BBPSO) [
Concepts related to chaos theory have been integrated with PSO to improve its performance. This type of PSO variant is called chaotic PSO (CPSO). Chuang et al. [
In order to make PSO more powerful, it was combined with fuzzy sets theory. This type of PSO variant is called fuzzy PSO (FPSO). Juang et al. [
PSO with timevarying acceleration coefficients (TVAC) was proposed to further improve the performance of standard PSO. The new variant was termed PSOTVAC. Cai et al. [
Oppositionbased learning (OBL) theory was integrated with PSO, and the new variant was dubbed oppositionbased PSO (OPSO). Dhahri and Alimi [
In contrast, some researchers objected to those researches that made PSO more and more complex, and they tended to simplify standard PSO without impairing its performance, with the aim of reducing computation time, improving convergence performance, or making it easier to implement. For example, Guochu [
The premature of PSO can be avoided by not using the entire swarm’s best known position
Some researchers make tentative research on improving the optimization performance of PSO by other efficient strategies. For example, Chuang et al. [
PSO was combined with some traditional and evolutionary optimization algorithms in order to take the advantages of both methods and compensate the weaknesses of each other. This type of PSO is called hybridized PSO.
Kuo and Hong [
Tang et al. [
Li et al. [
Chen and Chien [
Sait et al. [
ElAbd [
Maione and Punzi [
Xu et al. [
The success of the PSO algorithm as a singleobjective optimizer within continuous search space has motivated researchers to extend its use to other areas, which consist of but not limited to multiobjective optimization, constrained optimization, binary optimization, discrete optimization, combinatorial optimization, and so forth.
Multiple objective PSO (MOPSO) has been proposed to deal with multiobjective optimization problems, in which the objective function took Pareto dominance into account when moving the PSO particles and nondominated solutions were stored so as to approximate the Pareto front. Qiu et al. [
Scholars have proposed several solutions to constrained optimization problem. For example, Daneshyari and Yen [
Discrete PSO (DPSO) was proposed and harnessed to address discrete optimization/integer programming problems. Chen and Ludwig [
01 integer programming or binary integer programming (BIP) is the special case of integer programming where variables are required to be 0 or 1. Binary PSO (BPSO) was used to solve this type of problems. Zhai and He [
The choice of parameters of PSO has a large impact on its optimization performance [
PSO may converge to global optimal or local optimal positions. The latter is unexpected for PSOusers. Attempts at mathematically analyzing PSO convergence exist in literatures. These analyses offer guides for selection parameters of PSO in another way that guarantees global convergence and avoid premature. Wang and Shen [
Parallel computing is a computational form, in which computations are carried out simultaneously. On one hand, computers using multicore, multiprocessor, and graphics processing unit (GPU) contain multiple processing elements within a single machine. On the other hand, clusters, grids, and clouds employed multiple computers to work on the same task.
PSO can be implemented on multicore (multiprocessor) conditions. Waintraub et al. [
GPU is a specialized electronic circuit designed to rapidly manipulate and alter memory to accelerate the creation of images in a frame buffer intended for output to a display. Modern GPUs are very efficient at manipulating computation of PSO. Hung and Wang [
Cloud computing is a computing form in which large groups of remote servers are networked to allow centralized data storage and online access to computer services or resources. Scholars studied the performance of PSO implemented in cloud computing. Liu et al. [
PSO has been used in many applications in various academic and industrial fields so far. Using the analytical tool provided by “Web of Science Core Collection,” the hottest application categories are “electrical and electronic engineering,” “automation control systems,” “communication theory,” “operations research,” “mechanical engineering,” “fuel and energy,” “medicine,” “chemistry,” “biology,” and so forth.
PSO was used by researchers to solve the optimization problems encountered in electrical and electronic engineering. Ganguly et al. [
PSO has found several applications in automatic control systems. Cai and Yang [
An interesting application area of PSO is communication theory. Yousefi et al. [
PSOs have been employed to solve problems in operations research. Liu and Wang [
A few applications of PSO are related to mechanical engineering. Ming et al. [
PSO was widely used to solve the optimization problems existing in civil engineering. Ashuri and Tavakolan [
PSO had been introduced for the fuel and energy area. Mandal et al. [
In medicine, Qasem and Shamsuddin [
In chemistry, Thakral et al. [
Saraswathi et al. [
Revisiting Section
Adherence to SI principles.
Principle  Adherence of PSO 

Proximity principle  Particle swarm carried out 
Quality principle  Particle swarm responds to quality factors 
Diverse response principle  Particle swarm responses allocated between 
Stability principle  Particle swarm changes state only when 
Adaptability principle  Particle swarm changes state every time 
Like all other SIbased optimization approaches, PSO has some drawbacks like premature, high computational complexity, slow convergence, sensitivity to parameters, and so forth. The reasons behind the problems are complicated. One potential reason is that PSO does not utilize the crossover operator as employed in GA or DE; hence the distribution of good information between candidates is not at a required level. Another reason may fall within the fact that PSO does not appropriately handle the relationship between exploitation (local search) and exploration (global search), so it usually converges to a local minimum quickly.
To address abovementioned problems, scholars have proposed many solutions listed in Section
Major modifications, including quantumbehaved PSO, barebones PSO, chaotic PSO, fuzzy PSO, PSOTVAC, OPSO, SPSO, and topology.
Minor modifications, including constriction coefficient, velocity clamping, trap detection, adaptive parameter, fitnessscaling, surrogate modeling, cooperative mechanism, boundary shifting, position resetting, entropy map, ecological behavior, jumpingout strategy, preference strategy, neighborhood learning, and local search.
Hybridization, PSO being hybridized with GA, SA, TS, AIS, ACO, HS, ABC, DE, and so forth.
Different modifications of PSO algorithm.
Contributions  Studies 

QPSO  Jau et al. [ 


BBPSO  Zhang et al. [ 


CPSO  Chuang et al. [ 


FPSO  Juang et al. [ 


PSOTVAC  Cai et al. [ 


OPSO  Dhahri and Alimi [ 


SPSO  Guochu [ 


Population topology  Wang and Watada [ 


Other modifications  Chuang et al. [ 


PSOGA  Kuo and Hong [ 


PSOAIS  Tang et al. [ 


PSOTS  Li et al. [ 


PSOACO  Chen and Chien [ 


PSOSA  Sait et al. [ 


PSOABC  ElAbd [ 


PSODE  Maione and Punzi [ 


Other hybridization  Xu et al. [ 


Multiple objective  Qiu et al. [ 


Constrained optimization  Daneshyari and Yen [ 


Discrete optimization  Chen and Ludwig [ 


Binary optimization  Zhai and He [ 


Parameter choice  Kumar and Chaturvedi [ 


Convergence analysis  Wang and Shen [ 


Multicore  Waintraub et al. [ 


GPU  Hung and Wang [ 


Cloud  Liu et al. [ 
Now there are over hundreds of both various PSO variants and test functions at present. It is impossible for each newly proposed PSO variant to compare with all other variants and to go through all test functions. Therefore, it is difficult to proclaim which type of modification is better or promising. In our opinion, it is important and essential to create a platform, to which the authors who proposed PSO variants can submit their programs. After the comprehensive and fair comparison, we then can conclude which PSO variant is the winner. Particularly, the design of PSO without parameters is worth studying.
Success of PSO as a singleobjective optimizer for processing continuous optimization problem has motivated researchers to extend the use of PSO to other optimization fields, like multiobjective optimization, constrained optimization, and discrete optimization (binary optimization and combinatorial optimization).
For multiobjective optimization, the optimizer needs to consider Pareto dominance every time it updates particles and store nondominated solutions to approximate the Pareto front.
For constrained optimization, both infeasible and feasible solutions should be generated at the search stage, and constraints are dealt with when evaluating solutions using penalty factor
For discrete optimization, a commonly used method is to map the discrete space to a continuous domain and apply standard PSO, followed by demapping the results. However, that method is not efficient and costs high computational resources. Particular PSObased methods have been developed as discrete PSO, binary PSO, and their variants.
Study of PSO in those fields had impressive achievements; however, lack of indepth research on theoretical aspects impaired its application potentials. We believe it should be interesting to perform a more comprehensive theoretical study of both the runtime and convergence properties of PSO and its variants. Other aspects such as fitness landscapes and dynamics of PSO are also very attractive theoretical research directions.
For the programmers, writing parallel programs are more difficult than to write sequential programs, since they need to take into consideration the communication and synchronization between different subtasks. Therefore, there are merely a few publications related to PSO implemented in parallel. Some new computational forms are worth investigating within the next years, such as performing PSO in computer clusters, in grid computing, and in MPP (massively parallel processor).In addition to them, different computational levels of parallel computing are also interesting: bitlevel, instructionlevel, datalevel, and tasklevel.
The first practical application of PSO was in the field of ANN training and was reported together with the algorithm itself [
Application areas of PSO algorithm.
Area  Publication 

Electrical and electronic engineering  Ganguly et al. [ 


Automatic control  Cai and Yang [ 


Communication  Yousefi et al. [ 


Operations  Liu and Wang [ 


Mechanical engineering  Ming et al. [ 


Civil engineering  Ashuri and Tavakolan [ 


Fuel and energy  Mandal et al. [ 


Medicine engineering  Qasem and Shamsuddin [ 


Chemical engineering  Thakral et al. [ 


Biological engineering  Saraswathi et al. [ 
From the table, it is clear that the distribution of PSO exceeded our expectations. It is now applied to nearly every discipline that comes across optimization problems. Generally, PSO can be applied for optimization in dynamic and uncertain environments. The engineers need to first transform the problem to an optimization problem and then apply PSO to solve it.
Several research topics need to be fully investigated in future, since there are so far either no or merely few publications applying PSO to the optimization problem in those fields including what follows.
Symbolic regression, which is a type of regression analysis that searches the space of mathematical expressions to find the model that best fits a given dataset, both in terms of accuracy and simplicity, is a crucially important theoretical and practical problem.
Floorplanning is to design the layout of equipment in a factory or components on a computer chip to reduce manufacturing time.
Weapon target assignment problem is to find an optimal assignment of a set of weapons of various types to a set of targets in order to maximize the expected damage done to the opponent.
Supply chain management is the systemic, strategic coordination of the traditional business functions and the tactics across these business functions within a particular company and across businesses within the supply chain, for the purposes of improving the longterm performance of the individual companies and the supply chain as a whole [
Nurse scheduling problem [
Queuing theory is the mathematical study of waiting lines. In queueing theory a model is constructed so that queue lengths and waiting times can be predicted to make business decisions about the resources needed [
Figure
Publication number against year.
Artificial bee colony
Ant colony optimization
Artificial intelligence
Artificial immune system
Artificial neural network
Bat algorithm
Biogeographybased optimization
Barebones PSO
Braincomputer interface
Bacterial foraging optimization
Backpropagation
Binary PSO
Computational intelligence
Chaotic PSO
Compute unified device architecture
Discrete cosine transform
Differential evolution
Distributed generation
Discrete PSO
Evolutionary computation
Economic dispatch problem
Electroencephalography
Economic load dispatch
Electromyography
Fuzzy cmeans
Finite element analysis
Fuzzy PSO
Genetic algorithm
Genetic programming
Graphics processing unit
Gravitational search algorithm
Glowworm swarm optimization
Harmonic search
Hadoop distributed file system
Kernel SVM
Multivariable input and multivariable output
Maximum likelihood estimate
Many Optimizing Liaisons
Multiple objective PSO
Maximum power point
Magnetic resonance
Multicast routing problem
Minimal spanning tree
Oppositionbased learning
Oppositionbased PSO
Proportionalintegral derivate
Particle swarm optimization
PSO with TVAC
Photovoltaic
Quantumbehaved PSO
Radial basis function
Simulated annealing
Sequential backward selection
Sequential forward selection
Swarm intelligence
Simplified PSO
Sequential quadratic programming
Support vector machine
Support vector regression machine
Tabu search
Traveling salesman problem
Timevarying acceleration coefficients
Unmanned aerial vehicle
Unmanned combat aerial vehicle
Ultrawideband
Vehicle routing problem
Wavelet transform.
The authors declare that there is no conflict of interests regarding the publication of this paper.
This work was supported by NSFC (no. 610011024), Program of Natural Science Research of Jiangsu Higher Education Institutions of China (no. 14KJB520021), Jiangsu Key Laboratory of 3D Printing Equipment and Manufacturing (BM2013006), and Nanjing Normal University Research Foundation for Talented Scholars (no. 2013119XGQ0061).