Reference Function Based Spatiotemporal Fuzzy Logic Control Design Using Support Vector Regression Learning

This paper presents a reference function based 3D FLC design methodology using support vector regression (SVR) learning. The concept of reference function is introduced to 3D FLC for the generation of 3D membership functions (MF), which enhance the capabilityofthe3DFLCtocopewithmorekindsofMFs.Thenonlinearmathematicalexpressionofthereferencefunctionbased3D FLCisderived,andspatialfuzzybasisfunctionsaredefined.Viarelatingspatialfuzzybasisfunctionsofa3DFLCtokernelfunctions ofanSVR,anequivalencerelationshipbetweena3DFLCandanSVRisestablished.Therefore,a3DFLCcanbeconstructedusing thelearnedresultsofanSVR.Furthermore,theuniversalapproximationcapabilityoftheproposed3Dfuzzysystemisproven intermsofthefinitecoveringtheorem.Finally,theproposedmethodisappliedtoacatalyticpacked-bedreactorandsimulation resultshaveverifieditseffectiveness.


Introduction
Many industrial processes and systems are "distributed" in space [1] and are usually called spatially distributed systems.Recently, a novel three-dimensional fuzzy logic controller (3D FLC) [2] has been developed for the control of such systems.The 3D FLC employs a three-dimensional (3D) fuzzy set [2], which is composed of the traditional fuzzy set plus a third dimension for the spatial information, and carries out a 3D rule inference engine; thus, it has the inherent capability to process spatiotemporal dynamic systems.The control strategy of the 3D FLC is similar to how human operators or experts control the temperature in a space domain.Actually, it is a kind of spatiotemporal fuzzy control system with the traditional model-free advantage.
Currently, most 3D FLC designs are based on expert knowledge [2][3][4][5], which requires that the human knowledge to the control solution must exist and be structured [6].However, in many real-world applications, experts may have problems structuring the knowledge.Sometimes, although experts have the structured knowledge, they may sway between extreme cases: offering too much knowledge in the field of expertise or tending to hide their knowledge [6].On the contrast, data sets hidden with effective control rules are usually available.The motivation of this study is to design a 3D FLC using spatiotemporal data information.
To date, few literatures are found to be focused on databased 3D FLC design methods.In [7], a table look-up scheme was employed to design 3D FLC in terms of input-output pairs.In [8], a fuzzy c-means algorithm (FCM) and gradientdescent approach were used to design a data-based 3D FLC, where FCM was used to learn the initial 3D fuzzy rule base and then the gradient-descent approach was used to optimize the parameters of MFs.In [9,10], a clustering and linear support vector regression based 3D FLC design method was proposed, where the nearest neighborhood clustering was used to construct the antecedent part of 3D fuzzy rules and a linear support vector regression (SVR) was used to learn the consequent parameters.These methods either yield lots of fuzzy control rules (e.g. in [7]) or require additional algorithm to reduce redundant 3D fuzzy sets or 3D fuzzy rules [8][9][10].As a complementary, Zhang et al. proposed a data-based 3D FLC design method using SVR learning [11], where the learned support vectors and associated learning parameters are directly used to design antecedent part and consequent part of 3D fuzzy rules.The best advantage of this method is that reasonable 3D fuzzy control rules are directly extracted and constructed by SVR learning.The limitation of the design is that Gaussian shape membership function (MF) is the only choice for MF design.
In this study, we focus on a reference function based 3D FLC design using SVR learning, which integrates the merits of SVR learning and flexible MF choice.Utilizing the concept of reference function, the 3D FLC can cope with more kinds of MFs, for example, Symmetric triangle, Gaussian, Cauchy, Laplace, Hyperbolic Secant, and Squared Sinc.A nonlinear mathematical description of a reference function based 3D FLC can be derived, and spatial fuzzy basis functions are defined.Via relating spatial fuzzy basis functions of a 3D FLC to kernel functions (KFs) of an SVR, an equivalence relationship between a 3D FLC and an SVR is established.Therefore, a 3D FLC can be constructed using the learned results of an SVR.In addition, the universal approximation capability of the proposed 3D fuzzy system is proven in terms of the finite covering theorem.
The paper is organized as follows.Preliminaries about the reference function, 3D MF generated by reference function, and the nonlinear mapping of reference function based 3D FLC are addressed in Section 2. In Section 3, the methodology and design scheme of the reference function based 3D FLC design using SVR learning are presented.Then, the finite covering theorem is used to prove that the 3D FLC is a universal approximator in Section 4. In Section 5, a catalytic packed-bed reactor is presented as an example to illustrate the proposed 3D FLC and validate its effectiveness.In Section 6, conclusions are given.
Many functions may be reference functions.For instance, Symmetric triangle, Gaussian, Cauchy, Laplace, Hyperbolic Secant, and Squared Sinc as listed in Table 1 are reference functions.The reference functions can be used to generate 3D MFs, which provide a way for 3D FLC to access more kinds of 3D MFs.

3D MF Generated by Reference Function.
A 3D MF is an extension of a traditional MF by adding a third coordinate for the spatial information.In detail, the 3D MF has three coordinates: one is for the universe of discourse of the variable, another one is for the spatial information, and the third one is for the membership degree.If finite sensors are used, the 3D MF can be considered as the assembly of the traditional 2D MFs at each sensing location.In this way, Membership degree )), and  3 1 ( 1 ( 1 )) are generated by the Gaussian type reference function  1 ( 1 ( 1 )).
we can generate a 3D MF by location transformation of a reference function at each sensing location.

Reference Function Based 3D FLC as a Nonlinear
Mapping.The basic structure of a 3D FLC is composed of 3D fuzzifier, 3D rule inference, and defuzzifier.Due to its unique 3D nature, some detailed operations of a 3D FLC are different from a traditional one for spatial information expression, processing, and compression.For their detailed operations, one can refer to [2].Once each component of a reference function based 3D FLC is set, the nonlinear mathematical description of the 3D FLC can be derived (see Appendix A for a brief derivation).Assuming that we employ 3D singleton fuzzifier, 3D fuzzy rules as shown in (A.4) of Appendix A, "product" t-norm and "weighted aggregation" dimension reduction [3] in the 3D rule inference, singleton fuzzy sets for the output variable, and "linear" defuzzifier [14], the reference function based 3D FLC can be mathematically expressed as where   (  ) denotes the input of the th spatial input variable   () from the sensing location  =   ;    (  (  )) =   (  (  )−   ) denotes MF generated by the reference function   (  (  ));    denotes a location parameter, that is, the location transformation of the MF of   (  ) with respect to   (  (  ));   denotes the spatial weight from theth sensing location;  0 and   are constants;  denotes sensor number.
In (1), let then (1) can be rewritten as We define Ψ  (  ) as a Spatial Fuzzy Basis Function (SFBF) [11].Each SFBF corresponds to a 3D fuzzy rule, and all the SFBFs correspond to a 3D rule base.Mathematically, a 3D FLC is a linear combination of all the SFBFs.Furthermore, we rewrite (2) into (4) where   ((  )) = ∏  =1    (  (  )).From (4), we can find that, at each sensing location, there exists a traditional FBF [15]   ((  )); in the whole space domain, multiple traditional FBFs are assembled by the spatial weights  1 , . . .,   into a SFBF Ψ  (  ).All the spatial information expression and processing as well as the fuzzy linguistic expression and rule inference are integrated into SFBFs.
Equation (1) (or (3)) shows that the reference function based 3D FLC is a nonlinear mapping from the input space   ∈ Ω ⊂  × to the output space (  ) ∈  ⊂ .In particular, using (3) a reference function based 3D FLC can be represented by a three-layer network structure as show in Figure 2.

Reference Function Based 3D FLC Design
Using SVR Learning x 1 (z) x 2 (z) an SVR as the KFs, and the learned spatial support vectors as leading spatiotemporal data points from the SVR are imported for the design of a 3D fuzzy rule base.The design theory will involve two crucial issues.The first one is whether SFBFs from a 3D FLC can be used as KFs in an SVR.The second one is what the relationship between a 3D FLC and an SVR would be like on the basis of the first issue.

Spatial Fuzzy Basis Function as Mercer KF.
When relating the SFBFs with the KFs in an SVR, for instance, SFBFs are regarded as KFs, the SVR and the 3D FLC will have the same network structures and then have the same mathematical expressions, which will be discussed in Section 3.1.2.Generally speaking, a function satisfying Mercer theorem can be used as a KF for an SVR [16].In this study, we will prove that an SFBF is a Mercer KF.In (3), we rewrite Ψ  (  ) into (  ,   ), which can be further expressed as follows: where   ∈  × is a spatial input,   ∈  × is the location transformation parameter of 3D MF in the th rule, and   ((  ),    ) and (  ,   ) are translation invariant KFs [13].In terms of [13], if the reference functions are positive definite functions, then we do get a Mercer kernel.The reference functions as listed in Table 1 are positive definite functions.Using these reference functions to generate MF, from [13], we can conclude that   ((  ),    ) is a Mercer kernel.Since the linear combination of KFs is still a KF [16], we can derive that (  ,   ) is still a Mercer KF.Therefore, SFBFs are Mercer KFs, which can be used as KFs for SVR learning.

Mathematical Equivalence of a Spatial SVR and a 3D
FLC. Once the SFBFs from the 3D FLC are employed as the KFs for an SVR, an inherent equivalence relationship will be built between the SVR and the 3D FLC (comparing Figure 2 and Figure 10).By combining (1) and (B.7), we have the following mathematical expressions: where  0 = ,   = ( *  −   ), and Ψ  (  ) = (  ,   ) = (  ,    ).From (6), we can find that each spatial support vector    and its associated learning parameter ( *  −   ) correspond to one 3D fuzzy rule, where    is applied to set the center of the 3D MF of the 3D fuzzy set    ( = 1, . . ., ) in the th rule, that is, the location transformation of the 3D fuzzy set with respect to reference function (  ,   ), and ( *  −   ) is used to set   (the constant for the consequent set of the th rule in 3D FLC).

Design Scheme.
The design of a reference function based 3D FLC consists of five parts: data collection, KF generation, SVR learning, 3D fuzzy rule construction, and 3D fuzzy controller integration, as shown in Figure 4.
(1) Data Collection.A set of spatiotemporal data will be collected.The data should contain effective control laws.Essentially, the reference function based 3D FLC design is a fuzzy modeling [17] that extracts fuzzy control rules from the spatiotemporal data.
(2) KF Generation.Before SVR learning, KFs should be properly designed.In this step, via properly selecting reference function, SFBFs from a 3D FLC will be formulated (as in (4)) to set KFs for SVR learning.the spatial support vector    is employed to construct the antecedent part of the th rule;  *  −  is employed to construct the consequent part of the th rule.The form of each 3D fuzzy rule is shown as below It is shown that the result of the SVM learning can be easily interpreted using structured linguistic knowledge.Finally, a 3D rule base with  rules is established.
(5) 3D Fuzzy Controller Integration.Once the 3D rule base is established, a 3D FLC can be achieved by integrating other components including 3D fuzzifier, 3D rule inference, and defuzzifier.The detailed settings are given in Section 2.2.2.Finally, we obtain a complete 3D FLC, which can be used as a controller for a spatially distributed dynamic system.

Universal Approximation of Reference Function Based 3D FLC
The reference function based 3D FLC design method is used to construct a 3D FLC from spatiotemporal data hidden with effective control laws.In other words, the 3D FLC aims at approximating an unknown nonlinear control function.
In this study, we use the finite covering theorem to prove that the 3D FLC is a universal approximator; that is, it can approximate continuous control functions to arbitrary accuracy.
The universal approximation capability of the reference function based 3D FLC can be described by Theorem 1.
Theorem 1.Let g(  ) :  × →  be a continuous function defined on a compact Ω.For each  > 0, there exists a reference function based 3D FLC (  ) such that sup From (A.4), it is shown that  0 is an universal rule, namely, for any spatial input   ,  0 will be fired.In (A.4), the fired rule  0 will produce the constant  0 .Let     Before the proof of Theorem 2, we first present some preparation work.

𝐹(𝑥
When   =   ∈ Ω, the firing level of the fired rule   ( = 1, . . ., ) is The inference result of   is given by where  is the output variable of the 3D FLC, which is corresponding to the "" of the consequent part of the fired rule.The composition result of all the fired rules is given as follows: Based on the above preparation, Lemma 3 is presented as follows.

Lemma 3. Under the condition of
We can construct a 3D FLC (  ), defined by When   =   ∈ Ω, we have the following.

A Catalytic Packed-Bed
Reactor.This designed 3D FLC is applied to a catalytic packed-bed reactor [1,4,18] shown in Figure 5, where a reaction of the form  →  takes place on the catalyst.The reaction is endothermic and a jacket is used to heat the reactor.A dimensionless model that describes this nonlinear tubular chemical reactor is provided as follows: subject to the boundary conditions where   ,   , and  denote the dimensionless temperature, the concentration of reactant , and jacket temperature, respectively;  and  denote the dimensionless time and space;   and   are the heat and mass Peclet numbers,   is a dimensionless heat of reaction,   is a dimensionless preexponential factor,   is a dimensionless activation energy, and   is a dimensionless heat transfer coefficient.The values of the process parameters are given as follows: = 5.0,   = 5.0,   = 0.00001,   = 1.0,   = 15.62,  = 22.14.
The control problem is to maintain a desired reaction rate via tuning the jacket temperature to control catalyst temperature.In this application, the reactor began to work at one steady state; because of the requirement of operation conditions, the reference value of temperature is increased by 8%.Thus, the control objective is to make the temperature of reactor well track the new reference value along the space domain.
The detailed design of the expert based 3D FLC, including 3D fuzzifier, 3D rule inference, and defuzzifier, can refer to [4].The scaling factors for the spatial error, the spatial error in change, and the incremental output are set as 0.5, 0.1, and 0.3, respectively.The sample period is 0.1 s, and the sampling duration is 6 s.Thus, we have 60 input-output data pairs (shown in Figure 6), each of which is represented by ([()  , Δ()  ], Δ  ),  = 1, . . ., 60.
(2) KF Generation.The reference functions (Symmetric triangle, Gaussian, Cauchy, Laplace, Hyperbolic Secant, and Squared Sinc) as listed in Table 1 are employed, respectively, to generate 3D MFs and then formulate SFBFs.SFBFs are used for KFs in an SVR learning.
(3) SVR Learning.With the spatiotemporal data set as above, the SVM learning algorithm is used for the support vector learning.It should be noted that the SFBFs in (3) are taken as the KFs.As a result, spatial support vectors are extracted and their associated learning parameters are obtained.For instance, when the Gaussian type reference function is used for KF generation, eight support vectors were learned from 60 spatiotemporal data pairs when  = 1000,  = 0.00005, and  = 0.1, as shown in Figure 7, where the spatiotemporal input data is decomposed into multiple two-dimensional graphical representations over the space domain.
(4) 3D Fuzzy Rule Construction.In terms of the learned results of the SVR in the previous step, we establish 3D fuzzy rules.For instance, with the Gaussian type reference function, eight 3D fuzzy rules are constructed.The first four 3D fuzzy rules are presented as follows.
The first four 3D fuzzy rules and their associated 3D MF distributions can be depicted in Figure 8, which show the inherent spatial nature of the 3D fuzzy control system.
(5) 3D Fuzzy Controller Integration.Based on the 3D fuzzy rules established in step (4), we obtain a complete 3D FLC by combining 3D fuzzifier, 3D rule inference, and defuzzifier.The resultant 3D FLC will be used as a controller for the catalytic packed-bed reactor.

Control Performance
Validation.The designed reference function based 3D FLC using SVR learning is validated on the nonlinear catalytic packed-bed reactor.We employed six kinds of reference functions, that is, Symmetric triangle, Gaussian, Cauchy, Laplace, Hyperbolic Secant, and Squared Sinc, and finally produced six 3D FLCs.With the reference function based 3D FLC as the controller, the catalyst temperature varying with time and space, manipulated input, and the catalyst temperature at steady state are presented in Figures 9(a)-9(f).The control performance is given in Table 2, where steady-state error (SSE), integral of the absolute error (IAE), and integral of time multiplied by absolute error (ITAE) [2] are used as the performance criteria.In terms of Figures 9(a)-9(f) and Table 2, we can find that different reference functions will yield different control performance.In this application, Gaussian, Cauchy, Hyperbolic Secant, and Squared Sinc reference functions result in good control performance, while Symmetric triangle and Laplace reference functions lead to poor control performance.The results illustrate that KF will influence the control performance; thus, in the actual application, we should choose proper KF to design a 3D FLC.
As a comparison, we do another control experiment; that is, the expert knowledge-based 3D FLC from [4] is taken as the controller.As for its detailed design including 3D MF, 3D rule base, 3D inference, fuzzification, and defuzzification, one can refer to [4].The scaling factors for the spatial error, the spatial error in change, and the incremental output are set as 0.5, 0.1, and 0.3, respectively.The controlled profiles and control performance are given in Figure 9(g) and Table 2, respectively.
From Figure 9 and Table 2, we can find that with a proper reference function, the reference function based 3D FLC has nearly the same control performance as the expert knowledge-based 3D FLC.It means that the proposed spatial SVR learning method can well extract the control laws hidden in a spatiotemporal input-output dataset and formulate them in the form of 3D fuzzy rules.

Conclusions
A reference function based 3D FLC design methodology using SVR learning is proposed for spatially distributed dynamic systems.Utilizing the concept of reference function, the 3D FLC can access more kinds of 3D MFs, such as Symmetric triangle, Gaussian, Cauchy, Laplace, Hyperbolic Secant, and Squared Sinc.Based on the mathematical expressions of reference function based 3D FLC, we define spatial fuzzy basis functions and then find an equivalence relationship between a 3D FLC and an SVR by connecting spatial fuzzy basis functions in the 3D FLC to KFs in the SVR.On the basis of the equivalence relationship, a 3D FLC can be designed using the SVR learning; that is, the learned spatial support vectors as the optimal leading data points can be directly used for 3D fuzzy control rule generation.The proposed reference function based 3D FLC design can be carried out in five steps: data collection, KF generation, SVR learning, 3D fuzzy rule construction, and 3D fuzzy controller integration.Besides, the universal approximation capability of the proposed 3D fuzzy system is discussed.Finally, effectiveness of the proposed 3D FLC design methodology is validated on a catalytic packed-bed reactor.
Then, utilizing a weighted aggregation [3] dimension reduction operation, a 2D set   is obtained.Consider the following: Finally, traditional inference operation (Mamdani implication operation) and linear defuzzifier [14] are carried out successively.We have the nonlinear mathematical expression as follows: (A.8)

B. Mathematical Preliminaries of 𝜀-Support Vector Regression
In this study, we focus on -support vector regression (-SVR).To make the SVR nonlinear, we may map the input vector  ∈   into the vector V of a high-dimensional feature space, V = Θ(), where Θ represents a mapping   →   , and formulate a linear regression problem in this feature space, and then an optimization problem will be solved.The optimization problem can also be solved in a dual space.By introducing the Lagrange multipliers   and  *  , the primal optimization problem can be formulated in its dual form as follows: where the constant  is a design parameter chosen by the user, which determines the tradeoff between the complexity of (, ) and the approximate error.Solving the dual quadratic programming problem, we can find an optimal weight vector  and an optimal bias  of the regression hypersurface given in (B.The training pattern   with nonzero ( *  −   ) is called support vector (SV).
To avoid a direct mapping Υ(), the kernel trick is used.AKF (  ,   ), which satisfies the Mercer's theorem, is introduced as below: Let  1 ,  2 , . . .,   represent support vectors.The solution of the SVR can be described by a three-layer network structure as shown in Figure 10.

3. 1 .
Design Methodology.The design methodology can be depicted by Figure3.The SFBFs from a 3D FLC are input to .

Figure 2 :
Figure 2: Three-layer network structure of a 3D FLC.

Figure 3 :Figure 4 :
Figure 3: Design methodology of reference function based 3D FLC design using SVR learning.

Figure 9 :
Figure 9: Control performance comparisons.From left to right in (a)-(g): catalyst temperature varying with time and space, manipulated input, and catalyst temperature at steady state.

Figure 10 :𝛼
Figure 10: Three-layer network structure of an SVR.
Suppose that we have a training set  = {[  ,   ] ∈   × ,  = 1, . . ., } consisting of  pairs ( 1 ,  1 ), ( 2 ,  2 ), . .., (  ,   ), where the inputs are -dimensional vectors and the labels are continuous values.In -SVR, the goal is to find a function (, ) so that for all training patterns  has a maximum deviation  from the target values   and has a maximum margin.The -insensitive loss function is defined as follows: