Intuitionistic Fuzzy Kernel Matching Pursuit Based on Particle Swarm Optimization for Target Recognition

In order to overcome the long training time caused by searching optimal basic functions based on greedy strategy from a redundant basis function dictionary for the intuitionistic fuzzy kernel matching pursuit (IFKMP), the particle swarm optimization algorithm with powerful ability of global search and quick convergence rate is applied to speed up searching optimal basic function data in function dictionary.The approach of intuitionistic fuzzy kernel matching pursuit based on particle swarm optimization algorithm, namely, PS-IFKMP, is proposed. This algorithm is applied to the aerospace target recognition, which requires real-time ability. Simulation results show that, compared with the conventional approaches, the proposed algorithm can decrease training time and improve calculation efficiency obviously with almost unchanged classification accuracy, while the model has better sparsity and generalization. It is also demonstrated that this approach is suitable for the application requiring both accuracy and efficiency.


Introduction
The kernel matching pursuit (KMP) Classifier is a new classification technique proposed by Vincent and Bengio in 2002 [1].The basic idea of KMP originates from matching pursuit (MP) method of sign processing theory, a greedy constructive algorithm that approximates a given function by a linear combination of basis functions chosen from a redundant basis function dictionary, and can be seen as a form of boosting.When training a KMP Classifier, there are mainly three steps.First, it maps training data into a redundant basis dictionary by a given kernel function and then uses greedy algorithm to find a linear expansion of basis functions that are selected from a redundant dictionary of functions to minimize the loss function and, at last, adjusts the corresponding coefficients by backfitting algorithm.The linear expansion of basis functions is the target function that we seek.The performance of the KMP Classifier is comparable to that of support vector machine (SVM) for classification, while typically requiring far fewer support points [2,3].Compared with the other Kernel methods, KMP is very flexible and has no constraint on the form of kernels.It can be allowed to mix several shapes of kernels in one instance.Up to now, the theory of KMP has been successfully applied to a variety of fields, such as pattern classification [4,5], image recognition [6], and intrusion detection [7].
However, there is a special case in practical application; the importance degree (or threat degree) of one type is higher than the other targets, so we need to classify the appointed important target much more precisely and reduce the recognition accuracy of the other unimportant targets.For example, in missile defense battle, the requirement of recognition accuracy of the true warhead is higher than that of bait, debris, and so forth.The traditional kernel matching pursuit algorithm deals with all samples equally and the decision function gives a full scale consideration for all samples in order to minimize total recognition error.
Unfortunately, such algorithm cannot perform well in special cases.To circumvent the disadvantage, Li et al. [8] proposed a fuzzy kernel matching pursuit (FKMP) machine, which can classify the appointed important samples much more precisely according to the predefined importance of the data.However, this method sets the fuzzy factor on the basis of artificial experience, which may bring risks to the training process.Lei et al. [9] proposed intuitionistic fuzzy kernel matching pursuit (IFKMP) machine and expanded KMP algorithm into intuitionistic fuzzy field to solve the problem that some important data cannot be classified precisely by 2 Mathematical Problems in Engineering assigning the intuitionistic fuzzy parameters to different samples efficiently.Nevertheless, the IFKMP machine essentially uses MP greedy algorithm to find a linear combination of basis functions that are selected from a redundant dictionary of functions, so the problem of greatly computer burden has not been solved, and the computing time can be increased greatly with searching spaces increasing, which may be more serious when the dictionary has two or few functions [10].To overcome the above drawback, the optimization methods have been introduced into MP algorithm.An improved genetic algorithm (GA) is proposed to simplify the calculation of MP algorithm in [11]; however, due to GA's prematurity, the stable capability of learning machine is not good.In [12], a quantum genetic algorithm is introduced for modifying MP process; however, the computing time cannot reduce effectively due to the slow search speed of quantum genetic algorithm.In [13], a novel technique for KMP based on intuitionistic fuzzy -means (IFCM) was proposed, but IFCM is still a local optimization algorithm and reduces the recognition accuracy of KMP.Particle swarm optimization (PSO) proposed by Kennedy and Eberhart [14] is a global optimization algorithm.Since its appearance, for its powerful ability of global search and quick convergence rate, PSO has received more and more attention [15,16].Based on analysis above, this paper put forward a novel technique for intuitionistic fuzzy kernel matching pursuit based on particle swarm optimization (PS-IFKMP), where the greedy algorithm is replaced by PSO algorithm.In order to test the effect and validity of PS-IFKMP algorithm, three different datasets are used for simulation.The results are compared with those based on KMP and IFKMP.Experiments results validate the preference and efficiency of PS-IFKMP.

Intuitionistic Fuzzy Kernel Matching Pursuit
2.1.Matching Pursuit.Matching pursuit was introduced in the signal-processing community as an algorithm that decomposes any signal into a linear expansion of waveforms that are selected from a redundant dictionary of functions.
Given  noisy observations { 1 ,  2 , . . .,   } at {x 1 , x 2 , . . ., x  } and a finite dictionary D of functions in Hilbert space , we aim to find sparse approximations of { 1 ,  2 , . . .,   } that are expansion of the form f  = ∑  =1   g  (x), which minimize the squared norm of the residue The process of MP can be described as follows: it starts at stage 0 with R 0 = y and f 0 = 0 and recursively appends functions to an initially empty basis.Given   , we build f +1 = f  +  +1 g +1 , by searching for g +1 ∈ D and  +1 ∈ R that minimize the squared norm of the residue ( From above, we can see that MP actually adopts greedy algorithm.In each iterative process, MP finds the most correlative basis function in D with the current residue R  .
As the basis is appended continuously, f  can approach observations { 1 ,  2 , . . .,   } more and more accurately.However, algorithm will generally stop when the norm of the residue ‖R  ‖ 2 goes below a predefined threshold or the number of the bases reaches the maximum.

Kernel Matching Pursuit.
Kernel matching pursuit is simply the idea of applying the matching pursuit family of algorithms to problem in machine learning, using a kernelbased dictionary [5].Given a kernel function  : R  × R  → R, we construct the basis dictionary of KMP on the training data: Kernel method is enlightened in great part to the success of the SVM, but there are some differences between them.The kernel used in SVM must satisfy the Mercer condition, while kernel in KMP has no such restriction.Simultaneously, one can use more than one kernel when constructing the function dictionary of KMP.There are some kernels in common use: ) .

Intuitionistic Fuzzy
where   (x  ) = ∑  =1     (x  ) represents the estimated value ŷ of the th sample; then the squared norm of the residue can be written as According to MP algorithm, we can get Then, For any g ∈ D,  ∈ R that minimizes the squared norm of the residue ‖r +1 ‖ 2 is given by For this optimal value of , we have So g ∈ D that minimizes expression ( 9) is the one that minimizes (11); we can get and the corresponding coefficient  +1 is At every step in this algorithm, not only the set of basis functions g  but also their coefficients   are obtained.So, the approximate form of observations { 1 ,  2 , . . .,   } in th iteration is However, when appending   g  , the linear combination may not be optimal.So a backfitting method is also done by reoptimizing all coefficients  1 , . . .,   to minimize the target cost: Finally, we can get the approximation functions in regression: or decision function in classification: where {V} is the support vector set obtained by IFKMP algorithm.

The Selection of Intuitionistic Fuzzy Parameter.
In practice, the recognition accuracy of different targets is different according to the threat degree.Generally, higher threat degree of the targets needs higher recognition accuracy.Aiming at this fact, the selection algorithm of intuitionistic fuzzy parameter can be listed as follows.
Algorithm 1 (the selection algorithm of intuitionistic fuzzy parameter).
Input.Input is the type of sample   .
Output.Output is the intuitionistic fuzzy parameter (  ).
Step 1. Determine   as appointed important target or nonappointed target.
Step 2. Calculate the membership degree (  ) and nonmembership degree (  ); this paper uses Gaussian function as membership function, so where  and  denote width and center point, respectively, (  ) = 1−(  ) is denoted as nonhesitancy degree, and (  ) denote the uncertainty degree which must be predefined on the basis of practice use.
Step 3. Calculate the intuitionistic fuzzy parameter (  ) by the following formula: The intuitionistic fuzzy parameter can distinguish the appointed important target and nonappointed important target specifically and classify the appointed important samples much more precisely.Moreover, the total misclassification rate will still be lower even with a higher misclassification rate  of the nonappointed important target.

IFKMP Based on Particle Swarm Optimization
Theoretically IFKMP algorithm solves the problem that some important data cannot be classified precisely according to the predefined importance of the data efficiently, but its implement is a greedy algorithm.Greedy algorithm requires that every step of searching process be global optimal searching in the redundant dictionary of function in order to select best matching signal structure, from which the large amounts of computing time have often not suffered.Particle swarm optimization algorithm with powerful ability of global search and quick convergence rate solves the problem, by replacing greedy algorithm to find a linear expansion of basis functions.

Theoretical Analysis.
We notice that a bottleneck of the IFKMP algorithm is represented by the search of the current best element from the redundant dictionary to be added in the function expansion.Usually, this requires a full search over the whole dictionary and may necessitate a large number of floating point operations.An alternative is provided by the so-called Weak Greedy Algorithm (WGA) [17] which provides an approximation of the MP and related greedy algorithms.WGA and its different formulations have been analyzed in [17,18] and proofs of convergence of the algorithm exist under various conditions.Unlike classical greedy algorithm, WGA generates an approximate sequence as follows: where α+1 =  +1  +1 , and, clearly, for  +1 = 1, WGA retrieves the original algorithm.The sequence  = { 1 ,  2 , . . .,   } is named weakness sequence and it must obey some constraints for the algorithm to converge.While different conditions on  result in different guaranteed convergence rates, we will simply require that ∃ t > 0 such that ∀ ≥ 1,   > t, which ensures the convergence [18].These modifications imply that we are no longer forced to produce the global maximum at each iteration of the IFKMP algorithm but just an approximate optimal value.So we can use PSO algorithm to search the next element from the redundant dictionary instead of greedy algorithm.
Compared with greedy algorithm, the PSO algorithm may only find an approximate optimal value; we still can ensure the convergence of PS-IFKMP algorithm according to the theory of WGA.As a consequence, the performance of the decision function we get from PS-IFKMP algorithm is similar to the decision function from IFKMP algorithm under the same termination condition.Based on this point, we believe that PS-IFKMP algorithm can decrease computational complexity while classification accuracy remains statistically the same by using PSO algorithm to find a linear expansion of basis functions.

Particle Swarm Optimization.
Particle swarm optimization is an evolutionary optimization technique based on metaphors for social interaction and communication such as flocks of birds and schools of fish.This stochastic, populationbased approach has been proven effective for solving both continuous and discrete optimization problems.Each particle in a swarm, which is analogous to a bird in a flock or a fish in a school, moves around in  dimensional search space.Based on its own experience and that of the swarm, it moves toward the best position in the search space [19].
The position and velocity of particle  at iteration  are represented by X   and V   , which can be defined as , respectively, and the personal best (best) of particle  is represented by P   , which denotes the position of particle  with the best fitness value found so far and is defined as P   = (  1 ,   2 , . . .,    ).The global best (best) of all particles at iteration  is represented by P   , which denotes the best position of the particle with the best fitness value in the swarm found so far and is defined as P   = (  1 ,   2 , . . .,    ).The new velocity and position of particle  can be obtained by ( 20) and (21), respectively: where V   () represents velocity of particle  at iteration  with respect to the th dimension and V +1  is new velocity at iteration  + 1,    is position of particle  at iteration  with respect to the th dimension, and  +1  is new position at iteration  + 1.    is position value of the th best at iteration  with respect to the th dimension and    is position value of the best at iteration . is current iteration. 1 ,  2 are acceleration coefficients. 1 ,  2 are uniform random numbers between 0 and 1.  is the inertia weight.Moreover, in order to make the speed of the particle not too high, a speed threshold V max should be set.If V   > V max , then V   = V max , and if

Target Recognition Algorithm of IFKMP Based on PSO.
IFKMP algorithm uses greedy algorithm to find a linear combination of basis functions, that is, the decision function f  in classification from a redundant dictionary of functions to minimize the objective function .The PS-IFKMP algorithm first maps training data into a redundant basis dictionary D by a given kernel function and then sets the population size according to the scale of the training dataset.At last, the linear expansion of basis functions is found by using PSO algorithm.Because the larger the value of linear expansion of basis functions, the more effective the Classifier, formula ( 16) is selected as the fitness function.The detailed steps of PS-IFKMP algorithm are listed as follows.
Algorithm 2 (intuitionistic fuzzy kernel matching pursuit based on particle swarm optimization).
Output.Output is the decision function f  ().
Step 1 (initialization).Set the maximum iterations  and iterative threshold  of IFKMP, set the population size  and the maximum iterations iter max of PSO, and suppose generation  = 1,  = 1.
Step 4. Measure the fitness of each particle in the population by f  (  , g  ) = ∑  =1    g   .
Step 5. Update particles according to formulas (21) and ( 22); then update the best P   and best P   .
Step 7. Let y = y − f  , and if y ≥  and  < , go to Step 3 by using the updated y and solve the next basic function g  and weight coefficient   .
Step 8. Recompute the optimal set of coefficients  1 , . . .,   by backfitting algorithm; then output the linear combination of the best weight coefficients and the corresponding basic functions, which is also the decision function f  ().
The flowchart of Algorithm 2 is shown in Figure 1.

Parameters Setting of PS-IFKMP. The inertia weight
can be a positive constant or even a positive linear or nonlinear function of time.When  > 1.2, the velocity item becomes the main item in the search direction of the particle.It extends the search area and finds the global optimum.
When  is between 0.8 and 1.2, three factors, velocity, best, and best, affect the velocity calculation for both local search and global search.When  < 0.8, only the best and best affect the new velocity calculation, which converges to the local optimum.Therefore, the value of inertia weight  is a tradeoff between the global search and the local search.In this paper, the inertia weight starts with a high value  max and nonlinearly decreases to  min at the maximal number of iterations [20]: where  max and  min are the initial and final values of the inertia weight, respectively, iter denotes the current number of iterations, and iter max denotes the maximum number of iterations.The value  is a constant coefficient.The acceleration factors  1 and  2 are positive constants controlling the relative impact of the personal (local) and common (global) knowledge on the movement of each particle, and we improve the selection of acceleration factors by introducing inheritance mechanism: where f   is the fitness value of particle  at iteration  and the convergence rate of PS-IFKMP speeds up efficiently by introducing inheritance mechanism.
In order to keep the diversity of the population, the Gaussian distribution is introduced.If the P   has not been improved in a few steps, a certain proportion of particles should be randomly selected for mutation: where  0 is a Gaussian random number with mean of 0 and standard deviation of 1.  kernel function to construct the basis function dictionary and then searches the best basis functions and the corresponding weight in each iteration.So the computation complexity can be described by the number of multiplications in one matching process in this paper.

Complexity
Assuming that the size of dictionary is  and the iteration number is , IFKMP algorithm requires  ⋅  single-precision multiplications in one matching process, and the complexity can be described as ( ⋅ ) based on the computational analyses of Montgomery reduction algorithm.In PSO algorithm, position, velocity, and fitness of each particle need to be calculated in every iteration, so the algorithm requires 3 ⋅  ⋅  single-precision multiplications in one matching process, where  is the population size and the complexity of PS-IFKMP can be described as (3 ⋅  ⋅ ).As the population size  is far less than the dictionary size , the computational cost of PS-IFKMP is far smaller than that of IFKMP when  is quite big, while the costs are familiar when  is small.

Experiment Results and Analysis
This section evaluates the performance of the proposed PS-IFKMP algorithm by comparing its classification results with those obtained for the same cases by the KMP in [1] and IFKMP in [9] where PSO is encoded by real number and RBF kernel is used for both algorithms.To avoid the weak problem, each experiment has been performed at 50 independent runs.The average results and standard deviations (Std. in short) are given.All experiments were carried on a Pentium (R) Dual-Core CPU E5500 @2.8 GHz with 2 GB RAM using MATLAB 7.6 compiler.

Test on UCI Data.
In this experiment, five UCI datasets of Musk, Waveform, German, Diabetes, and Breast Cancer Wisconsin (Breast in short) are used to test our methods, where Waveform contains three classes, and two of them (class 0 and class 2) are selected as experimental dataset.The characteristics of the UCI datasets are shown in Table 1, which shows a wide range of attributes sizes and class sizes.
The selection of different kernel function parameters has great influence on algorithm performance, so this paper validates the kernel parameter by a special training dataset.The intuitionistic fuzzy parameters are set as ( 1 ) = ( 2 ) = 1,  = 200,  = 0.05,  = 30, and iter max = 50 and the kernel function parameter  is taken in [1,500] by equal interval sampling.The influence of  on the accuracy is shown in Figure 2.
From Figure 2, we can notice that the optimal kernel function parameters based on different datasets are different.Validated by experiments,  = 360, 3, and 6 for Musk, Waveform, and German, respectively.There is no special requirement on the importance of samples; we can set ( 1 ) = ( 2 ) = 1; then IFKMP algorithm is equivalent to KMP algorithm, so we only compare PS-IFKMP with the KMP in this section; the results are listed in Tables 2-4.
From the above experiments, the average accuracy of KMP and PS-IFKMP is 85.37% and 86.34%, respectively, so the performance of the PS-IFKMP is competitive to that of KMP for classification.However, the average training time in one matching process of KMP and PS-IFKMP is 0.16 and 0.048, respectively; compared with KMP, the average training time of PS-IFKMP is decreased by 70%.So the proposed PS-IFKMP algorithm can decrease obviously training time with almost unchanged classification accuracy, especially for the large size datasets.
Diabetes contains 500 negative samples and 268 positive samples, and Breast contains 241 negative samples and 458 positive samples.As we know, the positive samples describe the pathological features of diseases, so the positive samples should be classified much more precisely in terms of medical diagnosis.Validated by experiments,  = 15 and 3 for Diabetes and Breast, respectively.The intuitionistic fuzzy parameters are selected by Algorithm 1, ( 1 ) of appointed positive samples is 1.6, and ( 2 ) of nonappointed negative samples is 0.3.The results are listed in Table 5.
From Table 5, facing imbalanced training set, standard KMP cannot classify the appointed important samples efficiently.IFKMP solves the aforementioned problem by assigning the intuitionistic fuzzy parameters to different samples and studying the weak samples sufficiently.Compared with IFKMP, the proposed PS-IFKMP algorithm can decrease training time obviously, and the classification accuracy is almost unchanged simultaneously.

Test on Artificial Data.
Learning to tell two spirals apart is important both for purely academic reasons and for industrial application.In the research of the pattern recognition, it is a well-known problem for its difficulty.The parametric equation of the two spirals can be presented as follows: Spiral-1: where  1 ,  2 ,  1 , and  2 are parameters.In our experiment, we choose  1 =  2 = 4,  1 = 1, and  2 = 3.We generate 12000 samples randomly, and the samples distribution is shown in Figure 3(a).Moreover, a set of concentric circles samples also is selected to test our methods, and the parametric equation of the concentric circles data can be shown as Parameter  of the first class is of continuous uniform distribution [0, 55] and the second class  ∼  [45,100].16000 samples are generated randomly, and the samples distribution is shown in Figure 3(b).Setting  = 150,  = 0.02,  = 50, iter max = 50, and  = 8, the intuitionistic fuzzy parameters are selected by Algorithm 1; ( 1 ) of appointed important samples (shown by "•" in Figure 3) is 1.5 and ( 2 ) of nonappointed samples (shown by "×" in Figure 3) is 0.3.Before the training, noise is added to training data, randomly choosing 10% samples and changing its class attributes.The results are listed in Table 6.
In Table 6, it is easily found that KMP cannot classify the appointed important samples efficiently when training Classifier with a subset of the data.IFKMP can classify the appointed important samples much more precisely by assigning the intuitionistic fuzzy parameters to different samples and studying the weak samples sufficiently.The proposed PS-IFKMP with advantage of both PSO and IFKMP reduces the training time enormously with the classification accuracy almost unchanged.

Test on Aerospace Target Data.
In order to test the performance of the proposed PS-IFKMP algorithm in practical application, the aerospace targets data are selected for experiment.Generally, the threat degree of tactical ballistic missile (TBM), cruise missile (CM), and stealth aircraft (SA) is higher than the other targets, so we set these three targets as the appointed important targets.The radar cross     4.
Beside the appointed important targets, a large number of debris, baits, and other aircrafts, and so forth, are selected as negative samples.Setting  = 200,  = 0.05,  = 30, iter max = 50, and  = 3.5, the ( 1 ) of TBM equals 1.5, the ( 2 ) of CM equals 1.3, and the ( 3 ) of CM equals 1.6 by Algorithm 1.The results are listed in Table 7.
From Table 7, by providing equal treatment on all training samples, the classical KMP algorithm cannot classify the weak important data efficiently.The IFKMP circumvents this problem by studying the important samples sufficiently and the less important samples roughly; however, the complexity problem still exists by adopting the greedy algorithm.It is also demonstrated that this approach is suitable to the application requiring both accuracy and timeliness.

Conclusion
Theoretically IFKMP algorithm is an excellent method but its implement is based on a greedy algorithm.Therefore, the defect of great computer time has not been solved when dealing with huge volume of data.In order to overcome the limitation, we propose the PS-IFKMP algorithm which optimizes searching process based on particle swarm optimization and reduces computational complexity of IFKMP algorithm.Simulation results show that, compared with the conventional approaches, the proposed algorithm can decrease training time and improve calculation efficiency obviously with the classification accuracy almost unchanged, especially for large size datasets.However, further improvements are needed by this algorithm, and the future research should focus on problems such as how to provide better multiclass recognition performance and how to select the parameters of the kernel function.
Accuracy varying with  based on Musk data Accuracy varying with  based on Diabetes data Accuracy varying with  based on German data Accuracy varying with  based on Breast data

Figure 3 : 15 Figure 4 :
Figure 3: The distribution map of artificial data.
Given  noisy samples {(x 1 ,  1 , ( 1 )), . .., (x  ,   , (  ))}, where x  ∈ R  represents the sample feature,   ∈ R represents the sample observation, and (  ) is intuitionistic fuzzy parameter and given a kernel function  : R  × R  → R, we construct the basis dictionary on the training data: D = {g  = (⋅, x  ) |  = 1, . .., }.Then the residue is Analysis.The performance of the proposed PS-IFKMP algorithm is analyzed in theory in this section.
The IFKMP is a novel algorithm which was proposed in 2011, and no relevant improved algorithms have been proposed at present.Therefore we mainly compared theoretical complexity of the treatment of the proposed PS-IFKMP algorithm with IFKMP algorithm.IFKMP algorithm maps training data into a high-dimensional Hilbert space by a given

Table 2 :
Testing of Musk dataset.
a # s. v.: number of support vectors.

Table 3 :
Testing of Waveform dataset.

Table 4 :
Testing of German dataset.

Table 5 :
Testing of appointed important samples.

Table 6 :
Testing of artificial dataset.RCS) values of the target are simulated by FEKO software, and the maximum, minimum, mean, and variance of RCS sequence are selected as sample features.And the RCS sequence of TBM within 30 seconds is showed in Figure

Table 7 :
Testing of artificial dataset.