MPE Mathematical Problems in Engineering 1563-5147 1024-123X Hindawi 10.1155/2017/2427309 2427309 Research Article Robust Kernel Clustering Algorithm for Nonlinear System Identification http://orcid.org/0000-0002-6295-1739 Bouzbida Mohamed 1 2 http://orcid.org/0000-0003-2257-7436 Hassine Lassad 1 2 Chaari Abdelkader 1 2 Gordillo Francisco 1 National Higher Engineering School of Tunis (ENSIT) University of Tunis 5 Av. Taha Husein BP 56 1008 Tunis Tunisia utunis.rnu.tn 2 Laboratoire d’Ingenierie des Systemes Industriels et des Energies Renouvelables (LISIER) University of Tunis ENSIT Tunis Tunisia utunis.rnu.tn 2017 1452017 2017 12 12 2016 16 03 2017 30 03 2017 1452017 2017 Copyright © 2017 Mohamed Bouzbida et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

In engineering field, it is necessary to know the model of the real nonlinear systems to ensure its control and supervision; in this context, fuzzy modeling and especially the Takagi-Sugeno fuzzy model has drawn the attention of several researchers in recent decades owing to their potential to approximate nonlinear behavior. To identify the parameters of Takagi-Sugeno fuzzy model several clustering algorithms are developed such as the Fuzzy C-Means (FCM) algorithm, Possibilistic C-Means (PCM) algorithm, and Possibilistic Fuzzy C-Means (PFCM) algorithm. This paper presents a new clustering algorithm for Takagi-Sugeno fuzzy model identification. Our proposed algorithm called Robust Kernel Possibilistic Fuzzy C-Means (RKPFCM) algorithm is an extension of the PFCM algorithm based on kernel method, where the Euclidean distance used the robust hyper tangent kernel function. The proposed algorithm can solve the nonlinear separable problems found by FCM, PCM, and PFCM algorithms. Then an optimization method using the Particle Swarm Optimization (PSO) method combined with the RKPFCM algorithm is presented to overcome the convergence to a local minimum of the objective function. Finally, validation results of examples are given to demonstrate the effectiveness, practicality, and robustness of our proposed algorithm in stochastic environment.

1. Introduction

Modeling and identification are significant steps in the design of the control system. Typical applications of these models are the simulation, the prediction, or the control system design. Generally, the modeling process consists of obtaining a parametric model with the same dynamic behavior of the real process. However, when the process is nonlinear and complex, it is very difficult to define the different mathematical or physical laws which describe its behavior [1, 2]. In this context, the modeling of nonlinear systems by the conventional methods is very difficult and occasionally ineffective [3, 4]. So, other nonconventional techniques based on fuzzy logic are used more often in modeling this kind of process due to excellent ability of describing its behavior .

Among the best fuzzy modeling approaches developed in literature we mention the Takagi-Sugeno fuzzy model. In effect, this model is described by if-then rules. Each rule includes a fuzzy set antecedent and mathematical functions as consequent representing the process behavior in each region [2, 3, 7]. The identification problem consists of estimating the model parameters. In this context, to identify the parameters of Takagi-Sugeno fuzzy model, many techniques were developed such as the Adaptive schemes, heuristic approaches, nearest neighbor clustering, and support vector learning mechanisms. Besides fuzzy clustering algorithms are widely used in fuzzy modeling. Fuzzy C-Means (FCM), Gustafson-Kessel (G-K), Gath-Geva (G-G), Possibilistic c-Means (PCM), Fuzzy C-Regression Model (FCRM), Enhanced Fuzzy C-Regression Model (EFCRM), and Possibilistic Fuzzy c-Means (PFCM) are popular clustering algorithms used in structure identification part and LS, WRLS, and orthogonal least square (OLS) technique were applied in consequent parameter estimation. Among the clustering algorithms, Fuzzy c-Means (FCM) developed by Bezdek is well-known but this algorithm is sensitive to noise or outliers and susceptible to local minima [8, 9]. However, noise in the data sets can make the situation worse by creating many inauthentic minima. These are able to distort the global minimum solution found by FCM algorithm. This flaw has stimulated the researchers to overcome this inconvenience. To fight against the effects of outlying data, various approaches are considered such as the possibilistic clustering (PCM) proposed by Krishnapuram and Keller  and fuzzy noise clustering approach of Dave . The possibilistic approach executed a possibilistic partition, in which a membership relation calculates the absolute degree of typicality of a point in a cluster. Although the PCM algorithm is robust against the noise points and allows identifying these outliers, it is very responsive to initializations and occasionally generates coincident clusters. To solve this deficiency of identical clusters, a Possibilistic Fuzzy c-Means (PFCM) algorithm was suggested by Wu and Zhou in 2006. Nonetheless, these algorithms are not efficient for unequal dimension clusters and cannot separate clusters that are nonlinearly separable in input space and their limits between two clusters are linear. In our work to overcome this shortcoming, kernel methods  are regarded as the way of dealing with this problem. We propose a new clustering algorithm called Robust Kernel Possibilistic Fuzzy C-Means (RKPFCM), which adopts a kernel induced metric in the data space to replace the original Euclidean norm metric. By changing the inner product with an appropriate hyper tangent kernel function, one can implicitly affect a nonlinear mapping to a high dimensional feature space where the data is more clearly separable. However, our proposed algorithm RKPFCM has an iterative nature that makes it sensitive to initialization and sensitive to converge to a local minimum. To overcome these problems, several solutions have been proposed in the literature. Among them is combining the clustering algorithm with a heuristic optimization technique. In this context, many researches have proposed the evolutionary computation technique based on Particle Swarm Optimization (PSO). They have been successfully applied to solve various optimization problems . Thus, we introduce PSO into the RKPFCM algorithm to achieve global optimization. The efficacy of our algorithm compared to many algorithms is tested on noisy nonlinear systems defined by recurrent equations and an application to Box Jenkins system. This paper will be presented as follows. The second part of work is reserved for introducing the Takagi-Sugeno fuzzy model. The third part will be devoted to identifying the premise parameters of this model where we used the proposed RKPFCM algorithm and PSO algorithm is introduced. In the fourth part, we will focus on identification of consequent parameters. The simulations results and the model validity of RKPFCM and RKPFCM-PSO are presented in part five. Finally, we conclude this paper.

2. Takagi-Sugeno Fuzzy Model

Takagi-Sugeno fuzzy model (T-S) is one of the best techniques used for modeling a nonlinear system represented by the recurrent equation yk=gNL(xk). T-S model is constructed by a rule-based type “if … then” in which the consequent uses numeric variables rather than linguistic variables (case of Mamdani). The consequent can be expressed by a constant, a polynomial, or differential equation depending on the antecedent variables. The T-S fuzzy model allows approximating the nonlinear system into several locally linear subsystems [4, 17].

In general, a Takagi-Sugeno fuzzy model is based on if then rules of the form(1)Ri:ifx1kisAi1,,xnkisAinthenyi=aiTxk+bi.The “if” rule function defines the premise part and “then” rule function constitutes the consequent part of the T-S fuzzy model.

Ri: represents the ith rule;

aiT=ai1,ai2,,ain: is the parameters vector, such as aiRn;

bi: is a scalar;

xk=x1k,x2k,,xnkT: is observations vector;

Ai1,Ai2,,Ain: represents the fuzzy subsets,

where i1,,C.

Here, the fuzzy sets are represented by the following membership function (2)Aijxjk=exp-xjk-vij2σij20,1i=1,2,,C,j=1,2,,n, where vi=[vi1,vi2,,vij]TRn are centers and σi=[σi1,σi2,,σij]TRn is the width of the membership function.

The estimated output model is defined by the following equation :(3)y^k=i=1Cβikyik.As(4)βik=j=1nAijxjki=1Cj=1nAijxjkk=1,2,,N,so,(5)yk^=i=1cβikaiTxk+bi.

3. Identification Algorithm for Premise Parameters

To identify the premise parameters of a Takagi-Sugeno fuzzy model described by equation (1), we used the Possibilistic Fuzzy C-Means (PFCM) algorithm and our proposed algorithms (RKPFCM and RKPFCM-PSO).

3.1. Possibilistic Fuzzy <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="M31"><mml:mrow><mml:mi>C</mml:mi></mml:mrow></mml:math></inline-formula>-Means (PFCM) Algorithm

The Possibilistic Fuzzy C-Means (PFCM) algorithm, which uses Euclidean distance, finds the partition of the collection X={x1,,xN}Rp of N measures, specified by k-dimensional vectors xi=x1k,x2k,,xikT, into C fuzzy subsets by minimizing the following objective function :(6)JPFCMU,T,V=i=1Ck=1Naμikm+btikDik2+i=1Cηik=1Ntiklogtik-tik,where

1CN: the number of clusters;

μik: the membership of xk in cluster i satisfying (7)0μik1,i=1Cμik=11kN;

tik: the typicality of xk in classes i; (8)D2ik=xk-vi2;

V: the set of cluster centers (viRp);

ηi: the suitable positive numbers described by(9)ηi=Kk=1Nμik,FCMmD2ikk=1Nμik,FCMm,K>0.

Typically, K is chosen to be 1.

μik,FCM are the terminal membership values of FCM.

m is a weighting degree; this parameter has a significant impact on the form of clusters in data space.

To minimize equation (6), we take its partial derivative of variables, μik,tik, and vi, equal to zero and obtain the following equations:(10)μik=j=1CDikDjk2/m-1-1,(11)tik=exp-bDik2ηi,i,k,(12)vi=k=1Naμikm+btikxkk=1Naμikm+btik,i.

PFCM Algorithm Steps. Given a set of observations X=x1,,xNT.

Initialization (l=0)

Set the number of clusters C,1CN.

Set the level of weighting m: 2m<4.

Set the parameters  a, b.

Set the stopping criterion ε: ε>0.

Execute a FCM clustering algorithm to find initial fuzzy partition matrix U and cluster centers V.

Initialize the typicality matrix T randomly.

Compute ηi by (9).

Repeat for l = 1,2 , .

Step 1.

Compute the cluster centers by (12).

Step 2.

Compute the membership matrix U=μik by (10).

Step 3.

Compute the typicality matrix T=tik by (11).

Until U l - U l - 1 < ε ; then stop. Otherwise, set l=l+1 and return to Step  1.

3.2. Proposed Robust Kernel Possibilistic Fuzzy <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="M75"><mml:mrow><mml:mi>C</mml:mi></mml:mrow></mml:math></inline-formula>-Means (RKPFCM) Algorithm

The PFCM can deal with noisy data better than FCM and PCM; nevertheless, these conventional clustering algorithms become more effective when applied on linearly separable data or with a reasonable quantity of errors. In reality, the linearly separable data are rare. Therefore, FCM, PCM, and PFCM share the same negative point in that they are unable to get good separation of data that are nonlinearly separable in input space. To correct the imperfections found in PFCM particularly the nonlinear separable problem, kernel  methods are regarded as the way of dealing with this problem. In this context, we proposed a new extension of Possibilistic Fuzzy C-Means algorithm based on kernel method (RKPFCM). The present work proposes a way of increasing the accuracy of the PFCM algorithm by exploiting hyper tangent kernel function to calculate the distance used in its objective function.

The kernel function is defined as a generalization of the distance metric that measures the distance between two data points mapped into a future space in which the data are more clearly separable [12, 1921].

Define a nonlinear map as Φ:xΦ(x)F, where xX,andF is the transformed feature space with higher or even infinite dimension. X denotes the data space mapped into F .

The RKPFCM algorithm minimizes the following objective function:(13)JRKPFCMU,T,V=i=1Ck=1Naμikm+btikϕxk-ϕvi2+i=1Cηik=1Ntiklogtik-tik.Then xk-vi is mapped into space F :(14)xk-viϕxk-ϕvi,where(15)ϕxk-ϕvi2=ϕxk-ϕvi·ϕxk-ϕvi=ϕxk·ϕxk+ϕvi·ϕvi-2ϕxk·ϕvi=Kxk,xk+Kvi,vi-2Kkk,vi.K(xk,vi)=ΦxkTΦ(vi) is an inner product kernel function.

If we adopt the hyper tangent kernel function, that is, (16)Kxk,vi=1-tanh-xk-vi2σ2whereσ>0,then K(xk,xk)=K(vi,vi)=1. Thus (15) can be written as(17)ϕxk-ϕvi2=21-Kxk,vi=2tanh-xk-vi2σ2.Considering (17), the objective function (13) is transformed as follows: (18)JRKPFCMU,T,V=2i=1Ck=1Naμikm+btiktanh-xk-vi2σ2+i=1Cηik=1Ntiklogtik-tik.The derivation of the objective function (18) according to μik, ti, and vi, defines the relationship update of cluster centers and membership coefficients.

(i) Derivative of J RKPFCM ( U , T , V ) with respect to v i .(19)JRKPFCMU,T,Vvi=2k=1Naμikm+btik1-tanh2-xk-vi2σ2xk-viσ2=2k=1Naμikm+btik1-tanh-xk-vi2σ21+tanh-xk-vi2σ2xk-viσ2.So,(20)JRKPFCMU,T,Vvi=2k=1Naμikm+btikKxk,vi1+tanh-xk-vi2σ2xk-viσ2.Equating (20) to zero leads to(21)JRKPFCMU,T,Vvi=0.Then, (22)vi=k=1Naμikm+btikKxk,vi1+tanh-xk-vi2/σ2xkk=1Naμikm+btikKxk,vi1+tanh-xk-vi2/σ2.

(ii) Derivative of J RKPFCM ( U , T , V ) with respect to μ i k . In this part we used the Lagrange multiplier(23)JRKPFCMU,P,V=2i=1Ck=1Naμikm+btiktanh-xk-vi2σ2+i=1Cηik=1Ntiklogtik-tik-k=1Nλk·i=1Cμik-1,(24)JRKPFCMU,T,Vλ=-i=1Cμik-1=0,(25)JRKPFCMU,T,Vμik=2maμikm-1tanh-xk-vi2σ2-λ=0.From expression (25), we can write μik in this form:(26)μik=λ2matanh-xk-vi2/σ21/m-1.Substituting expression (26) in expression (24): (27)JRKPFCMU,T,Vλ=j=1Cμjk=j=1Cλ2matanh-xk-vj2/σ21/m-1=1.It is also(28)λk2ma1/m-1=1j=1C1/tanh-xk-vj2/σ21/m-1.The two expressions (26) and (28) give the following expression:(29)μik=1j=1Ctanh-xk-vi2/σ2/tanh-xk-vj2/σ21/m-1.Therefore the updating relationship is(30)μik=j=1C1-Kxk,vi1-Kxk,vj1/m-1-1.

(iii) Derivative of J RKPFCM ( U , T , V ) with respect to t i k .(31)JRKPFCMU,T,Vtik=2btanh-xk-vi2σ2+ηilogtik=0,(32)tik=exp-2btanh-xk-vi2/σ2ηi,i,k.Therefore, the updating typicality matrix is(33)tik=exp-2b1-Kxk,viηi,i,k.Similarly (9) is rewritten by(34)ηi=Kk=1Nμik,FCMm21-Kxk,vik=1Nμik,FCMm.

RKPFCM Algorithm Steps. Given a set of observations X=x1,,xNT.

Initialization (l = 0)

Set the number of clusters C,1CN.

Set the level of weighting  m: 2m<4.

Set the parameters a, b, and σ.

Set the stopping criterion ε: ε>0.

Execute a FCM clustering algorithm to find initial fuzzy partition matrix U and cluster centers V.

Initialize the typicality matrix T randomly.

Compute ηi  by (34).

Repeat for l = 1,2 , .

Step 1.

Compute the cluster centers by (22).

Step 2.

Compute the membership matrix U=μik by (30).

Step 3.

Compute the typicality matrix T=tik by (33).

Until U l - U l - 1 < ε ; then stop. Otherwise, set l=l+1 and return to Step  1.

3.3. Robust Kernel Possibilistic Fuzzy <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="M132"><mml:mrow><mml:mi>C</mml:mi></mml:mrow></mml:math></inline-formula>-Means Algorithm Based on PSO (RKPFCM-PSO) 3.3.1. PSO

The Particle Swarm Optimization is a heuristic search method proposed by Kennedy and Eberhart (1995). This technique uses random population solution particles to find an optimal solution to problems. Each particle moves in the search space with a dynamically adjusted position and velocity for the best solution. The particle is characterized by data structure that contains the coordinates of the current position in the search space, the best solution point visited so far, and the subset of other agents that are seen as neighbors. These adjustments are based on the historical behaviors of itself and other agents in the swarm. The change of speed (acceleration) and the position of each particle in the optimization landscape (search space) are iteratively [6, 12, 23](35)vkl+1=Kwvkl+ρ1pkl-xkl+ρ2pgl-xkl,xkl+1=xkl+vkl+1,where

k=1,,NP: size of particles;

D: size of the landscape (search space);

vk=(vk1,,vkd,,vkD): the speed of particle;

xk=xk1,xk1,,xkd,,xkD: the position of particle;

pk=pk1,pk1,,pkd,,pkD: the best previous position of particle;

g: index represents the best particle among all particles in the group;

K: the constriction factor described by the following relationship:(36)K=22-φ-φ2-4φ,whereφ=c1+c2,

c1 and c2 are two positive constants satisfying the following relationship: (37)φ=c1+c2>4;

ρ1 and ρ2: random variables defined as follows: (38)ρ1=r1×c1,ρ2=r2×c2,

r1 and r2 are two random variables between 0 and 1;

w: the weight of inertia according to this equation:(39)w=wmax-wmax-wminitermax×iter,

where wmax and wmin are the initial and final weight, itermax is the maximum iterations, and iter is the current iteration number.

3.3.2. Fitness Function

The fitness function defines our optimization problem described by the following expression:(40)Fitness=GJRKPFCMU,T,V,where

G is a positive constant.

JRKPFCM(U,T,V) represents the objective function of the RKPFCM algorithm.

RKPFCM-PSO Algorithm. Given a set of observations X=x1,x2,,xNT, the RKPFCM-PSO algorithm is described by the following steps.

Initialization (l=0)

Select the number of clusters C, fuzzy degree m, the parameters a and b, the population size NP, the constants c1 and c2, the random variables r1 and r2, the weight of inertia wmax and wmin, the size of the search space D, the constant G, and the stopping criterion ε.

Set the 1st particle generation clusters centers.

Initialize the fitness function and speed of each particle.

Compute ηi  by (34).

Repeat l = l + 1

Step 1.

Compute the fuzzy partition matrix U=μik by (30).

Step 2.

Compute the typicality matrix T=tik by (33).

Step 3.

Calculate the new value of fitness for each particle using (40).

Step 4.

Compare the fitness of each particle with pbest, if the value is better than pbest and then set the pbest value.

Step 5.

Compare the fitness value of gbest with the following: if the value is better than gbest, gbest then is set equal to this value.

Step 6.

Update position and speed of each particle by (35).

So this algorithm is converged when vk(l+1)-vk(l)<ε; that is to say, stop iteration and find the best solution in the last generation. If not, go back to Step1.

4. Identification for Consequent Parameters

The defuzzification method, used in the Takagi-Sugeno fuzzy model, is linear with the consequent parameters θi=aiT,bi which can be obtained as a solution of a weighted least squares problem according to the following equation:(41)θi=XeTψiXe-1XeTψiY,where

Xe=X;1 represents an extension of regression matrix;

X=X1T,X2T,,XNTT;

Y=y1,y2,,yNT is the output vector;

ψi is a diagonal matrix of dimension (N × N) containing the coefficients μik of fuzzy memberships.

The RKPFCM and RKPFCM-PSO clustering algorithm are used to find width of the membership functions by the following equation :(42)σij=2k=1Nμikxjk-vij2k=1Nμik.

5. Simulation Results and Validation Model 5.1. Identical Data with Noise

In this example we have used 12 data which are composed of 10 models and two noises; this data set (X12) is presented in . The FCM, PCM, PFCM, KPFCM, and our algorithms RKPFCM and RKPFCM-PSO were used in clustering the data set in tow groups (C=2).

In this example the parameters settings are a=1, b=2, m=2, G=10, NP=20, D=2, wmax = 0.9, wmin = 0.4, c1=2.05, c2=2.05,  σ = 20, itermax=1000, and  ε = 10−9.

Figure 1 shows the clustering results for our proposed method RKPFCM-PSO. The Ideal (true) centroids are(43)Videal=-3.3403.340.Table 1 shows the results of center clusters using the six algorithms. The error between the results prototypes and ideal center clusters is calculated by the next expression:(44)E=Videal-V2,where is the FCM, PCM, PFCM, KPFCM, RKPFCM, and RKPFCM-PSO.

Prototypes results of clusters centers.

Algorithms Centers V Error (E)
FCM  - 2.98 0.54 2.98 0.54 0.414
PCM  - 2.15 0.02 2.15 0.02 1.416
PFCM  - 2.84 0.36 2.84 0.36 0.3796
KPFCM  - 3.33 0.03 3.33 0.03 0.0005
RKPFCM - 3.337 0.002 3.337 0.002 1.79 × 10 - 5
RKPFCM-PSO - 3.3385 0.0001 3.3385 0.0001 4.23 × 10 - 6

Clustering by RKPFCM-PSO algorithm.

According to Table 1, our proposed algorithm RKPFCM -PSO gives the best prototypes of centers.

Figure 1 shows the effectiveness of our approach as well.

5.2. Identification of T-S Fuzzy Model

After applying the identification algorithm, it is necessary to validate the Takagi-Sugeno fuzzy model. Several validation tests of the model are used. Among them, we cited the Mean Square Error (MSE) test, Root Mean Square Error (RMSE), and the Variance Accounting For (VAF) test.(45)MSE=1Nk=1Nyk-yestk2,RMSE=1Nk=1Nyk-yestk2,VAF=100%1-vary-yestvary, where “y” is the real output and “yest” is the estimated output.

5.2.1. Example  1

Consider a nonlinear system described by the following difference equation :(46)yk=y2k-3+y2k-2+y2k-1+tanhuk1+y2k-1+y2k-2+y2k-3+ek,where y(k) and u(k) are the output and the input of the system, respectively.

e ( k ) is a noise. (47)uk=0.6sin3πkTs+0.2sin4πkTs+1.2sinπkTs.Ts = 0.01.

200 samples were generated by simulation and were used, where the selected input variables are chosen {y(k-1), y(k-2), y(k-3), u(k)}.

The complete data set has been used to train the model. The noise influence is analyzed with different SNR levels (SNR = 10 dB and SNR = 5 dB).

In this part, we have applied various algorithms and our proposed clustering RKPFCM and RKPFCM-PSO which approximate the nonlinear model (46).

The used parameters are a=1, b=3, m=2, G=10, NP = 30, D=4, wmax=0.9, wmin=0.4, c1=2.05, c2=2.05, σ = 20, itermax=1000, and ε = 10−9.

The shape of the excitation signal used for identification is illustrated in Figure 2.

Input-output sequences.

The simulation result given by the RKPFCM-PSO algorithm is illustrated in Figure 3.

Identification results for the RKPFCM-PSO algorithm with SNR = 10 DB.

Table 2 shows the various modeling performance results without noise obtained by different algorithms; this comparison results demonstrate that the best MSE and best VAF are obtained by the proposed methods (RKPFCM and RKPFCM-PSO).

Comparison results without noise.

Algorithms Number of rules MSE VAF (%)
Cluster fuzzy  6 1.9 × 10−3
Kalman cluster fuzzy  2 2.2 × 10−5
FCM 2 2.1422 × 10−5 99.9581
GK 2 2.1142 × 10−5 99.9592
KFCM 2 2.1011 × 10−5 99.9601
PFCM 2 2.0152 × 10−5 99.9611
RKPFCM 2 1.8498 × 105 99.9723
RKPFCM-PSO 2 1.8412 × 105 99.9742

Tables 3 and 4 present the various modeling performance results with noise influence (SNR = 5 dB and 10 dB) obtained by the different algorithms. However, our proposed algorithm RKPFCM-PSO retained the best performance with a higher level of noise.

Comparison results with SNR = 10 dB.

Algorithms Number of rules MSE (10−4) VAF (%)
FCM 2 9.8397 98.0345
GK 2 9.8831 98.0303
KFCM 2 9.9492 98.0289
PFCM 2 9.8313 98.0362
RKPFCM 2 8.5158 98.1628
RKPFCM-PSO 2 8.5149 98.1637

Comparison results with SNR = 5 dB.

Algorithms Number of rules MSE (10−3) VAF (%)
FCM 2 1.7594 96.0655
GK 2 1.7552 96.0668
KFCM 2 1.7572 96.0659
PFCM 2 1.7534 96.0783
RKPFCM 2 1,7424 96.1462
RKPFCM-PSO 2 1,73310 96.1510

The local linear models identified are given as follows:(48)y1k=0.3167yk-1+0.0263yk-2-0.0482yk-3+0.1183k+0.6916,y2k=0.4045yk-1+0.0665yk-2-0.0304yk-2+0.1519uk+0.5417.

5.2.2. Example  2

Consider a highly complex modified nonlinear system described by the following difference equation :(49)yk=yk-1yk-2+2yk-1+2.510+y2k-1+y2k-2+uk+ek,where y(k) is the model output and u(k) is the model input which is bounded between [−1+1]. The e(k) is a noise.

The following input signal is expressed as(50)uk=sin2kπ250k5000.8sin2kπ250+0.2sin2kπ25otherwise.1500 samples were generated by simulation in which 1000 samples were used to train the model. Fuzzy model parameters have been identified once, testing of model was done by the remaining 500 samples, and {y(k-1), y(k-2), u(k), u(k-1)} are chosen as input variables. In this example the parameters settings are a=1, b=2, m=2, G=10, NP = 30, D=4, wmax = 0.9, wmin = 0.4, c1=2, c2=2.1, σ = 20, itermax=1000, and ε = 10−9.

The noise influence is analyzed with different SNR levels (SNR = 20 dB, SNR = 10 dB, SNR =5 dB, and SNR = 1 dB).

The obtained identification results by RKPFCM-PSO are, respectively, shown in Figures 4 and 5.

Input-output sequences.

RKPFCM-PSO performances for test data with SNR = 10 dB.

The evaluation performance index (RMSE-trn and RMSE-test) stands for training and testing data, respectively.

Tables 58 show the comparative performance of RKPFCM and RKPFCM-PSO with different existing algorithms such as FCM, G-K, Fuzzy Model Identification (FMI), FCRM, and MFCRM-NC. It is clearly seen from the results that our algorithm RKPFCM-PSO gives the best performance in noisy environments. The local linear models identified by RKPFCM-PSO are given as follows:(51)y1k=0.8562yk-1+0.0548yk-2+2.0967uk-1.6609uk-1+0.0814,y2k=0.8058yk-1-0.0924yk-2+1.3488uk-0.3718uk-1+0.8032,y3k=0.5673yk-1+0.1896yk-2+1.2330uk-0.8613uk-1+0.0520,y4k=0.8881yk-1-0.0571yk-2+1.5400uk-0.9195uk-1+0.4536.

20 dB Noise.

Algorithms RMSE-trn RMSE-test Number of rules
FCM  0.1393 0.3004 4
GK  0.1326 0.1484 4
FMI  0.1457 0.2378 4
FCRM  0.1291 0.1495 4
MFCRM-NC  0.1183 0.1395 4
RKPFCM 0.1167 0.1377 4
RKPFCM-PSO 0.1165 0.1370 4

10 dB Noise.

Algorithms RMSE-trn RMSE-test Number of rules
FCM  0.3453 0.7150 4
GK  0.3330 0.3489 4
FMI  0.3956 0.4416 4
FCRM  0.3312 0.3694 4
MFCRM-NC  0.3233 0.3465 4
RKPFCM 0.3199 0.3405 4
RKPFCM-PSO 0.3186 0.3384 4

5 dB Noise.

Algorithms RMSE-trn RMSE-test Number of rules
FCM  0.6170 1.0364 4
GK  0.5855 0.6505 4
FMI  0.6274 0.7021 4
FCRM  0.5805 0.6485 4
MFCRM-NC  0.5658 0.5998 4
RKPFCM 0.4958 0.5921 4
RKPFCM-PSO 0.4908 0.5908 4

1 dB Noise.

Algorithms RMSE-trn RMSE-test Number of rules
FCM  0.9330 1.3646 4
GK  0.8932 1.0226 4
FMI  0.8953 1.0320 4
FCRM  0.8872 1.0095 4
MFCRM-NC  0.8699 0.9947 4
RKPFCM 0.7600 0.8894 4
RKPFCM-PSO 0.7571 0.8809 4
5.2.3. Example  3

We consider the Box Jenkins gas furnace data set which is used as a standard test for identification techniques. The data set is composed of 296 pairs of input-output measurements. The input “u” is the gas flow rate into a furnace and the output “y” is the CO2 concentration in the outlet gases. In order to take all the above-mentioned issues into account, we simulated the following experimental case :

all the 296 data pairs are used as training data and {y(k-1), u(k-4)} are selected as input variables to various algorithms, while we use two rules (C=2). In this example the used parameters are a=1, b=2, m=2, G=10, NP = 30, D=3, wmax = 0.9, wmin = 0.4, c1=1.95, c2 = 2.1,σ = 10, itermax=1000, and ε = 10−9.

The simulation result given by the RKPFCM-PSO algorithm is illustrated in Figure 6.

Identification results for the RKPFCM-PSO Algorithm (SNR = 10 dB).

Based on the comparison presented in Table 9, it is clear that the proposed algorithm RKPFCM-PSO is more robust to noise than the other algorithms found in literature.

Comparison results for Box Jenkins system.

Algorithms Number of inputs Number of rules MSE
Tong (1980)  2 19 0.469
Pedrycz (1984)  2 81 0.320
Xu (1987)  2 25 0.328
Sugeno (1991)  2 2 0.359
Yoshinari (1993)  2 6 0.299
Joo (1997)  2 6 0.166
Zhang (2006)  2 2 0.160
Glowaty (2008)  2 2 0.391
Andri (2011)  2 10 0.167
Tanmoy Dam (2015) 
Without noise 2 2 0.152
SNR = 30 dB 2 2 0.153
SNR = 10 dB 2 2 0.277
RKPFCM
Without noise 2 2 0.1458
SNR = 30 dB 2 2 0.1512
SNR = 10 dB 2 2 0.2342
SNR = 5 dB 2 2 0.2922
SNR = 2 dB 2 2 0.3014
RKPFCM-PSO
Without noise 2 2 0.1455
SNR = 30 dB 2 2 0.1503
SNR = 10 dB 2 2 0.2338
SNR = 5 dB 2 2 0.2902
SNR = 2 dB 2 2 0.3001

When we use our algorithm RKPFCM-PSO the local linear models identified are given as follows:(52)y1k=0.6777yk-1-0.9593uk-4+17.3007,y2k=0.5560yk-1-1.3297uk-4+23.5880.

6. Conclusion

In literature, various clustering algorithms have been proposed for nonlinear systems identification. In this work, we developed a new clustering algorithm called RKPFCM-PSO for the nonlinear systems identification. Our algorithm is an improvement of the Possibilistic Fuzzy C-Means Clustering (PFCM) where we used a hyper tangent kernel function to calculate the distance of data point from the cluster centers and a heuristic search algorithm PSO to reach the global minimum of the objective function. The proposed algorithm provides better results of fuzzy modeling of unknown nonlinear systems. The robustness and the quality of this proposed method are demonstrated by simulation results of noisy nonlinear systems described by recurrent equations and application to a Box Jenkins gas furnace system. Thus, the proposed methods show favorable results in noisy environments compared with the techniques mentioned in the literature.

In the future, we will integrate other optimization methods such as the gravitational search algorithm to optimize our hybrid method and we will apply this algorithm for identification of some complex nonlinear real systems as the robotic or the mechatronic systems.

Conflicts of Interest

The authors declare that they have no conflicts of interest.

Kim E. Park M. Kim S. Park M. A transformed input-domain approach to fuzzy modeling IEEE Transactions on Fuzzy Systems 1998 6 4 596 604 Sugeno M. Kang G. T. Fuzzy modelling and control of multilayer incinerator Fuzzy Sets and Systems 1986 18 3 329 345 2-s2.0-0022700023 10.1016/0165-0114(86)90010-2 Bezdek J. C. Pattern Recognition with Fuzzy Objective Function Algorithms 1981 New York, NY, USA Plenum Press 10.1007/978-1-4757-0450-1 Sugeno M. Yasukawa T. A fuzzy-logic-based approach to qualitative modeling IEEE Transactions on Fuzzy Systems 1993 1 1 7 31 10.1109/tfuzz.1993.390281 2-s2.0-0027544110 Chen J.-Q. Xi Y.-G. Zhang Z.-J. A clustering algorithm for fuzzy model identification Fuzzy Sets and Systems 1998 98 3 319 329 10.1016/S0165-0114(96)00384-3 MR1640915 Ahmed T. Lassad H. Abdelkader C. Nonlinear system identification using clustering algorithm and particle swarm optimization Scientific Research and Essays 2012 7 13 1415 1431 Abonyi J. Babuska R. Szeifert F. Modified Gath-Geva fuzzy clustering for identification of Takagi-Sugeno fuzzy models IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) 2002 32 5 612 621 10.1109/TSMCB.2002.1033180 Frigui H. Krishnapuram A robust competitive clustering algorithm with applications in computer vision IEEE Transactions on Pattern Analysis and Machine Intelligence Computation 1999 21 5 450 465 10.1109/34.765656 Ying K. C. Lin S. W. Lee Z. J. Lee I. L. A novel function approximation based on robust fuzzy regression algorithm model and particle swarm optimization Applied Soft Computing 2011 38 2 1820 1826 10.1016/j.asoc.2010.05.028 Krishnapuram R. Keller J. A possibilistic approach to clustering IEEE Transactions on Fuzzy Systems 1993 1 2 98 110 10.1109/91.227387 2-s2.0-0027595430 Dave R. Characterization and detection of noise in clustering Pattern Recognition Letters 2011 12 11 657 664 10.1016/0167-8655(91)90002-4 Zhang J. Shen L. An improved fuzzy c-means clustering algorithm based on shadowed sets and PSO Computational Intelligence and Neuroscience 2014 2014 10 10.1155/2014/368628 368628 Niknam T. Amiri B. An efficient hybrid approach based on PSO, ACO and k-means for cluster analysis Applied Soft Computing 2010 10 183 197 10.1016/j.asoc.2009.07.001 Tushir M. Srivastava S. A new Kernelized hybrid c-mean clustering model with optimized parameters Applied Soft Computing Journal 2010 10 2 381 389 2-s2.0-70849125660 10.1016/j.asoc.2009.08.020 Bidyadhar S. Debashisha J. A difierential evolution based neural network approach to nonlinear identification Applied Soft Computing 2011 11 1 861 871 10.1016/j.asoc.2010.01.006 Dam T. Kanti Deb A. Block sparse representations in modified fuzzy c-regression model clustering algorithm for ts fuzzy model identification Proceedings of the IEEE Symposium Series on Computational Intelligence December 2015 Cape Town, South Africa IEEE 1687 1694 10.1109/SSCI.2015.237 Takagi T. Sugeno M. Fuzzy identification of systems and its applications for modeling and control IEEE Transactions on Systems, Man and Cybernetics 1985 15 1 116 132 Wu X. H. Zhou A possibilistic c-means clustering algorithm based on kernel methods Computer Design and Applications (ICCDA '10) International Conference June 2006 Guilin, China IEEE 2062 2066 Kannan S. R. Ramathilagam S. Deviand R. Sathya A. Robust kernel FCM in segmentation of breast medical images Expert Systems with Applications 2011 38 4382 4389 10.1016/j.eswa.2010.09.107 Kaur P. Lamba I. M. S. Gosai A. Novel kernelized type-2 fuzzy c-means clustering algorithm in segmentation of noisy medical images Proceedings of the Recent Advances in Intelligent Computational Systems (RAICS '11) November 2011 Trivandrum, India IEEE 10.1109/RAICS.2011.6069361 Mohamed B. Ahmed T. Lassad H. Abdelkader C. A new extension of fuzzy c-means algorithm using non euclidean distance and kernel methods Proceedings of the Control, Decision and Information Technologies (CoDIT '13), International Conference May 2013 Hammamet, Tunisia IEEE 10.1109/CoDIT.2013.6689551 2-s2.0-84893281156 Zhao X. Zhou J. Improved kernel possibilistic fuzzy clustering algorithm based on invasive weed optimization Journal of Shanghai Jiaotong University 2015 20 2 164 170 10.1007/s12204-015-1605-z Eberhart R. C. Shi Y. Comparing inertia weights and constriction factors in particle swarm optimization Proceedings of the Congress on Evolutionary Computation July 2000 La Jolla, CA, USA, 84 88 10.1109/CEC.2000.870279 Babuka R. Verbruggen H. Neuro-fuzzy methods for nonlinear system identification Annual Reviews in Control 2003 27 73 85 10.1016/S1367-5788(03)00009-9 Pal N. R. Pal K. Keller J. M. Bezdek J. C. A possibilistic fuzzy c-means clustering algorithm IEEE Transactions on Fuzzy Systems 2005 13 4 517 530 2-s2.0-26844532803 10.1109/TFUZZ.2004.840099