A SVR Learning Based Sensor Placement Approach for Nonlinear Spatially Distributed Systems

Many industrial processes are inherently distributed in space and time and are called spatially distributed dynamical systems (SDDSs). Sensor placement affects capturing the spatial distribution and then becomes crucial issue to model or control an SDDS. In this study, a new data-driven based sensor placement method is developed. SVR algorithm is innovatively used to extract the characteristics of spatial distribution froma spatiotemporal data set.The support vectors learned by SVR represent the crucial spatial data structure in the spatiotemporal data set, which can be employed to determine optimal sensor location and sensor number. A systematic sensor placement design scheme in three steps (data collection, SVR learning, and sensor locating) is developed for an easy implementation. Finally, effectiveness of the proposed sensor placement scheme is validated on two spatiotemporal 3D fuzzy controlled spatially distributed systems.


Introduction
Many industrial processes are inherently distributed in space and time, such as fluid flow process, spray deposition process, heat exchange process, and snap curing process.These systems are usually called spatially distributed dynamical systems (SDDSs) or distributed parameter systems (DPSs) [1].Traditionally, the spatially distributed nature is ignored, and then an SDDS is simplified into a lumped parameter system (LPS).However, its performance will deteriorate if the system dynamics significantly vary with space [2].To satisfy tighter product quality requirements, the spatial nature should be considered in the modeling and control.Consequently, the spatiotemporal modeling and control for SDDSs have become the highlight in the modern modeling and control theory.
On the research of SDDS, since sensor placement affects capturing the spatial distribution, it becomes one of key issues to influence the performance of spatiotemporal modeling or control.Sensor placement is a difficult problem, particularly in nonlinear and multivariable case [3].
In the past several decades, sensor placement has been studied and can be classified into two categories, that is, model-based method and data-driven based method.For the first category, the sensor placement is dependent on an accurate mathematical model.Some methods aim at achieving better parameter estimation with different criteria, for example, scalar criteria of the covariance matrix [4] and scalar measure of performance defined on the Fisher information matrix [5].Some methods intend to improve state estimation or state observer with possible criteria, such as observability measures [6], convergence properties of observer [7], and cost function related to Gramian observability matrix [8].Some methods integrate optimal actuator and sensor placement with nonlinear output feedback control [9][10][11] and improve control performance, where the penalty on the response of the closed-loop system is used for cost function of actuator placement and the control action, and the estimation error in the closed-loop infinite-dimensional system is used for cost function of sensor placement.
For the second category, only data information is used for sensor placement.Wouwer et al. [12] proposed the determinant of Fisher information matrix formed by sensitivity functions for parameter estimation and the Gram determinant formed by sensor responses for observer estimation.Zamprogna et al. [13] selected the most suitable secondary process variable as soft sensor inputs for batch distillation by exploiting the properties of principal component analysis (PCA) on the sensitivity matrix.Tongpadungroda et al. [14] identified the optimal position of an applied load by GA under the criterion related to PCA.Zhang et al. [15] proposed a sensor placement method for spatiotemporal 3D fuzzy control system based on spatial constrained fuzzy -means algorithm.Compared with the model-based method, the datadriven method is very promising in practical engineering applications since no mathematical model is used.
In this study, a new data-driven based sensor placement method is proposed.Support vector regression (SVR) algorithm is used to extract the main characteristics of spatial distribution from a spatiotemporal data set, which can be directly used for the sensor placement.The support vectors learned by SVR represent the crucial spatial data structure hidden in the spatiotemporal data set, which can be employed to determine the optimal sensor location and sensor number.A systematic sensor placement design scheme is developed for an easy implementation, which consists of data collection, SVR learning, and sensor locating.The effectiveness of the proposed sensor placement scheme is validated on two spatiotemporal 3D fuzzy controlled nonlinear spatially distributed systems.
This paper is organized as follows.Preliminaries about nonlinear spatially distributed systems and SVR algorithm are described in Section 2. In Section 3, the SVR learning based sensor placement method is presented in detail.Two illustrative examples are given in Section 4. Finally, Section 5 summarizes the conclusion.

Nonlinear Spatially Distributed Dynamical Systems.
Many industrial processes exhibit highly nonlinear behavior and strong spatial variations.They usually can be represented by nonlinear partial differential equations.Some processes have strong convection characteristics [16], some have strong diffusion phenomena [17], and some have both convection and diffusion characteristics [18].These systems are usually installed with multiple spatially distributed actuators and multiple sensors and give rise to nonlinear control problems that involve the regulation of highly distributed control variables using these spatially distributed actuators and sensors.
In the following subsections, two examples of SDDSs are presented and illustrated.

A Catalytic Packed-Bed
Reactor.The catalytic packedbed reactor [17,19] in Figure 1 is a typical spatiotemporal dynamic system.A reaction of the form  →  takes place on the catalyst.The reaction is endothermic and a jacket is used to heat the reactor.The dimensionless mathematical model that describes this nonlinear tubular chemical reactor is given as follows: subject to the following boundary conditions: where   (, ) and   (, ) denote the dimensionless temperature of gas and catalyst, respectively, which are spatially dependent on  ∈ [0, 1], ()() denotes the spatiotemporal heating source with the distribution () and the manipulated input (), and () denotes the dimensionless temperature of jacket.The values of the process parameters are given as follows: In this application, the control target is to control the catalyst temperature   (, ) throughout the reactor in order  to maintain a desired degree of reaction rate using the measurements of catalyst temperature from  sensing locations   = [ 1  2 ⋅ ⋅ ⋅   ] and manipulating one spatially distributed heating source (() = 1 − cos()).The spatial reference profile is given as   () = 0.42 − 0.2 cos(), 0 ≤  ≤ 1, and then the entire spatial catalyst temperature should follow this reference.

A Rapid Thermal Chemical
Vapor Deposition System.Consider a rapid thermal chemical vapor deposition (RTCVD) reactor with three-zone heating banks (Adomaitis 1995;Theodoropoulou et al., 1998).The sketch of the RTCVD system is depicted in Figure 2, where the lamp bank  mainly heats the total area of the wafer, the lamp bank  mainly heats the wafer edge, and the lamp bank  is used to coarsely adjust the wafer temperature average.When a wafer is being heated, it is rotated for azimuthal temperature uniformity.10% silane feed in inert gas is inputted into the reactor from the top, and the silane is decomposed into silicon and hydrogen.A 0.5 m film of polysilicon is expected to deposit on the wafer when deposition temperature appears at near 800 K or higher.The temperature is controlled by operating the power to lamp banks , , and .A dimensionless wafer thermal dynamics equation (Adomaitis 1995) over the wafer domain is given as follows: subject to the boundary condition where    =   / amb denotes dimensionless wafer temperature,   denotes the actual wafer temperature, and  amb = 300 K denotes ambient temperature;   = /  denotes the dimensionless radial position of the wafer with  denoting the actual radial position of the wafer and   denoting the wafer radius (  = 7.6 cm);   = / denotes dimensionless time,  denotes the actual time, and  = 2.9 s denotes residence time of upper chamber;   (  ),   (  ), and   (  ) denote the radiant energy flux on the radial position   from the lamp banks , , and  to the wafer, respectively (the distributions are shown in Figure 3);   ,   , and   denote the percentage of the lamp power for the three-zone lamp banks.The values of the process parameters are given as follows: From ( 4) and ( 5), we can find that the wafer temperature is a variable varying with time and space.The concerned control problem is to control the wafer temperature throughout the wafer radius to arrive at the set temperature 1000 K quickly and uniformly using the measurements of wafer temperature from  sensing locations   = [ 1  2 ⋅ ⋅ ⋅   ] and manipulating power percentage (  ,   , and   ) to three zones of lamp banks.

SVR Algorithm. Support vector regression (SVR) is a
learning technique that originated from theoretical foundations of the statistical learning theory [20].Over the past decade, SVR has become one of the most popular regression techniques and has been employed in a variety of applications.It performs a new inductive principle (structural risk minimization) for learning from finite training data sets and chooses a model of the right complexity from a large number of candidate models (learning machines) to describe training data pairs.Suppose we have a training set  = {[  ,   ] ∈   × ,  = 1, . . ., } consisting of  pairs ( 1 ,  1 ), ( 2 ,  2 ), . . ., (  ,   ), where the inputs are -dimensional vectors and the labels are continuous values.SVR builds a function as follows: where  is a bias term and (⋅) :   → ℘ is an application mapping the space of factors into a feature space ℘.
The regression problem can be formulated as a convex optimization problem as follows: where  is a design parameter chosen by the user, which determines the trade-off between the complexity of (, ) and the approximate error, and [⋅]  is the -insensitive loss function which is defined as follows: In practice, ℘,  ∈ ℘ and () ∈ ℘ are deduced from a kernel (⋅, ⋅) :   ×   →  involved in the computation of a scalar product: The above optimization problem can be solved in a dual space.By introducing the Lagrange multipliers   ( *  ), the primal optimization problem can be formulated in its dual form as follows: Solving the dual quadratic programming problem, we can find an optimal weight vector  and an optimal bias  of the regression hypersurface given in Then, the best regression hypersurface is given by The training pattern   with nonzero ( *  −  ) is called support vector (SV).

SVR Learning Based Sensor
Placement Scheme   a maximum deviation  from the target values   and have a maximum margin.The support vectors, as crucial data points that represent principle spatial structures, are employed to determine the optimal sensor location and sensor number.

Systematic Design Scheme.
The systematic sensor placement design scheme is composed of three steps shown in Figure 5, including data collection, SVR learning, and sensor locating.Firstly, a set of data is collected as the fundamental data for sensor locating.To acquire sufficiently rich information from the system, the system should be excited by adding persistently exciting perturbation signals.Secondly, SVR is used to extract crucial data points from the collected data set.Thirdly, the sensor location is determined in terms of the extracted crucial data.
(1) Data Collection.The idea data should contain sufficiently rich information of dynamic characteristics.To acquire the sufficiently rich data, one possible method is to add persistently exciting perturbation signals to the input u().In this study, we use white noise and the pseudorandom multilevel signals (PRMSs) with maximal length [21] as the input perturbation signals for two application examples, respectively.When input perturbation signals are sequentially added to the input u(), the system output is collected.The collected data and its corresponding spatial location consist of fundamental data set S for the subsequent support vector learning.(2) SVR Learning.Before the learning, Gaussian kernel function with its spread  is selected and two learning parameters  and  are required to be set.We use -Fold Cross-Validation method to acquire , , and .Root mean-squared error (RMSE) is employed as quantitative performance criteria for test, which is defined as follows: where  *  denotes the predicted spatial point after learning,   denotes the practical spatial point, and  denotes the number of sensors.
(3) Sensor Locating.After the second step, we can obtain some data points that represent the spatial structure implied in the data set.In this step, we can easily determine the sensor locations, namely, the spatial locations in these support vectors.The number of support vectors is equal to the number of sensors.

Case Studies
In this section, we take the two practical applications described in Section 2 as examples.The mathematical model ( 1) and ( 2) in the catalytic packed-bed reactor and the mathematical model ( 4) and ( 5) in the RTCVD system are only for the process simulation for evaluation of the sensor placement scheme.The method of lines [22] is used to simulate the two models.

The Catalytic Packed-Bed
Reactor.The pseudorandom quinary signal (PRQS) with maximum length of 624 is used as the perturbation signal.The parameters of PRQS are chosen as follows: the lowest and uppermost values of input are 0.1474 and 0.0632, respectively, the number of the levels is 5, the sampling time is 0.1, the length of the period is 624, and the minimum switching time (clock period) is 0.1.The perturbation signal generated by PRQS is depicted in Figure 6.
The space domain of the system is discretized uniformly into 81 points; that is,  = [ 1 ,  2 , . . .,  81 ], the sampling period is set as 0.1, and the time interval of simulation is Then, the SVR is used to learn support vectors from the data set S. As for model selection problem of the SVR, we employ 5-Fold Cross-Validation method to select proper  from [2 2 , 2 15 ] with step-size set as 10,  from [2 −10 , 2 2 ] with step-size set as 0.01, and  from { 0 /100,  0 /95, . . .,  0 /( + 5),  0 /,  0 /(−5), . . .,  0 /10,  0 /5,  0 }, where  0 is set as 10% of the length of the input interval of (, ) denoted by where  max (  ) and  min (  ) are the maximum and the minimum bound value of temperature measurements from the th spatial input variable, respectively.In this application,  0 = 0.03969.
The effectiveness of the proposed sensor locating method is validated on the 3D fuzzy controlled catalytic packed-bed reactor.The design of 3D FLC is given in Appendices A and B. We carried out two different sensor placement schemes on the 3D fuzzy controlled catalytic packed-bed reactor.One is that two to nine sensors are evenly distributed in the space domain, respectively.The other is the proposed sensor locating scheme.The control performance comparison of the two schemes is given in Table 1, where SSE, IAE, and ITAE [23] stand for steady-state error, integral of the absolute error, and integral of time multiplied by absolute error for spatiotemporal dynamic systems, respectively.Graph comparison is given in Figure 7, where eight sensors in (a) are placed in the proposed optimal locations, while eight sensors in (b) are evenly placed.In terms of Table 1 and Figure 7, we can find that the proposed sensor placement scheme has improved the control performance.
In this application, the sample interval is 0.5 s, and the simulation duration is 100 s.The evolution profiles of manipulated variables are shown in Figure 8, and the measured wafer temperature (, ) is shown in Figure 9.
The effectiveness of the proposed sensor placement method is validated on the space decomposition and coordination based 3D fuzzy controlled RTCVD system.The design of space decomposition and coordination based 3D FLC is given in Appendix C. We carried out two different sensor placement schemes on the 3D fuzzy controlled RTCVD system.One is that three to eleven sensors are evenly distributed in the space domain, respectively.The other is the proposed sensor placement scheme.The control performance comparison of the two schemes is given in Table 2. Graph comparison is given in Figure 10, where six sensors in (a) are placed in the proposed optimal locations, while nine sensors in (b) are evenly placed.In the case of six sensors located at the proposed optimal locations, that is, {0.507, 1.52, 3.29, 4.81, 6.08, 7.60}, the RTCVD system is decomposed into three subsystems with one distributed control source, that is,   and sensing locations {0.507, 1.52, 3.29} for subsystem 1,   and sensing locations {3.29, 4.81, 6.08, 7.60} for subsystem 2, and   and sensing locations {4.81, 6.08, 7.60} for subsystem 3.In the case of nine sensors located at evenly placed locations, that is, {1/9, 2/9, 3/9, 4/9, 5/9, 6/9, 7/9, 8/9, 1}, the  decomposed three subsystems are given as follows:   and sensing locations {1/9, 2/9, 3/9, 4/9} for subsystem 1,   and sensing locations {3/9, 4/9, 5/9, 6/9, 7/9} for subsystem 2, and   and sensing locations {5/9, 6/9, 7/9, 8/9, 1} for subsystem 3.In terms of Table 2 and Figure 10, we can find that the proposed sensor locating scheme has improved the control performance in the RTCVD system.

Discussions.
According to the experiment results in the two examples, we can find that the proposed sensor placement scheme outperforms all evenly distributed sensor placement scheme.The results validate the effectiveness of the proposed sensor placement method in terms of the spatiotemporal 3D fuzzy control system.

Conclusion
The sensor placement for an unknown nonlinear SDDS is necessary for various applications (e.g., modeling and control design).In this paper, a data-based SVR learning based sensor placement method was proposed.The systematic design method is divided into three steps.Firstly, sufficiently rich data information is generated by adding persistently exciting perturbation signals and is collected as fundamental data for sensor placement.Secondly, SVR is used to extract the characteristics of spatial distribution that is expressed as support vectors.Thirdly, the learned support vectors are employed to determine optimal sensor location and sensor number.Simulations were presented to illustrate the effectiveness of this sensor placement method and its potential for a wide range of SDDSs.

A. Introduction of 3D FLC
Three-dimensional fuzzy logic controller (3D FLC) [23,24] is a novel rule-based fuzzy logic controller developed for SDDSs.It is based on a 3D fuzzy set (see Figure 11) for the spatial information and is based on a 3D inference engine to deal with spatial information.The central idea of the 3D FLC is to emulate the human operators' knowledge or expert experience to control a temperature field from the point of view of overall space domain.Similar to the traditional FLC, the 3D FLC still consists of fuzzification, rule inference, and defuzzification as shown in Figure 12.Due to its unique 3D nature, some detailed operations of the 3D FLC are different from the traditional one for spatial information expression, processing, and compression.The basic concept and basic components of the 3D FLC are introduced briefly as follows (refer to [23] for the detailed explanation).
(1) Spatial Input Variable and 3D Fuzzy Set.Spatial input variable, as an extension of the traditional input variable, is the function of the spatial coordinates and represents that the input information comes from the overall space domain.
In actual application, finite point sensors can be used for measurement; therefore, the input information of spatial input variable comes from the sensors located in the space domain.
The 3D fuzzy set is an extension of the traditional fuzzy set by adding three coordinates for the spatial information.As shown in Figure 11, the 3D fuzzy set has three coordinates: one is for the universe of discourse of the variable, another is for the spatial information, and the third is for the membership degree.If finite sensors are used, this 3D fuzzy set can be considered as the assembly of the traditional 2D fuzzy sets at each sensing location.
(2) Fuzzification.The fuzzification involving the spatial dimension will map crisp spatial inputs into 3D fuzzy inputs.There are two types of fuzzifiers: singleton fuzzifier and nonsingleton fuzzifier, whose detailed definitions are given in [23].If finite sensors are used, this 3D fuzzification can be considered as the assembly of the traditional 2D fuzzification at each sensing location.
(3) Rule Inference (i) Rule Base.Rules represent control strategy and human experience.For a 3D FLC with two spatial input variables, the th rule in the rule base can be expressed as follows: where () and Δ() denote scaled spatial input variables for error and error in change, respectively;  (ii) Inference Engine.As the kernel of 3D FLC, the inference engine is able to process spatial information and realizes two main functions: one is for overall behavior capture from spatial domain and the other is for traditional fuzzy inference.These two functions are realized in three following operations: spatial information fusion, dimension reduction, and traditional inference operation as shown in Figure 12.The spatial information fusion operation will fuse information at each spatial point and ultimately form a spatial membership distribution for each fired rule.The dimension reduction operation is to compress the 3D spatial distribution information into 2D information for each fired rule.Different method can be designed for this operation according to different control and design requirement.In this paper, a centroid approach will be employed to represent the overall behavior of the spatial domain.
(4) Defuzzification.After the 3D inference operation, a traditional fuzzy output is produced.Then, the traditional defuzzifier can be used to yield a crisp control action.Numerous candidates can be chosen for engineering applications, such as maximum, mean-of-maxima, centroid, center-of-sums, height, modified height, and center-of-sets.
Generally speaking, once fuzzy set and rule base are designed, the 3D FLC is determined.To reduce the complexity to tune the controller, scaling factors can be added to the inputs and output.

B. 3D FLC Design for the Catalytic Packed-Bed Reactor
The error of spatial catalyst temperature and its error change are taken as two spatial inputs for the 3D FLC; that is,  * () = { (1) 3D Fuzzy Set.Since 3D fuzzy set can be regarded as the assembly of traditional 2D fuzzy set from each sensing input, the design of 3D fuzzy set is transformed to the design of 2D fuzzy set.The 2D fuzzy set for each sensing input can be chosen as triangular shape as shown in Figure 13, where each input is classified into seven linguistic labels as positive large (PL), positive middle (PM), positive small (PS), zero (O), negative small (NS), negative middle (NM), and negative large (NL).
(2) 3D Fuzzification.Since 3D fuzzification can be regarded as the assembly of the traditional 2D fuzzification at each sensing location, the design of 3D fuzzification is transformed to the design of traditional 2D fuzzification.In this study, singleton fuzzification is used.
(3) 3D Rule Base.The linear control rule base is used; for instance, one of the rules is expressed as "if () is PM and Δ() is NB then Δ is NS," where PM and NB are 3D fuzzy sets, which are assembled by 2D fuzzy sets PM and NB at each sensing location; Δ is the incremental control action, whose fuzzy set is triangular in shape and classified into seven linguistic labels as shown in Figure 13; NS is the 2D fuzzy set.And the rule weight is defaulted as unity.
(4) 3D Rule Inference.The spatial -norm in spatial information fusion operation is chosen as "minimum."The centroid approach is used for the dimension reduction operation.In traditional inference operation, "minimum" and "maximum" are used for the -norm in the intersection operation and for the -conorm in the union operation, respectively.
(6) Spatial Scaling Factors.Scaling factor for each  *  is set to be 1.5, the scaling factor for each Δ *  is set to be 0.5, and the scaling factor for Δ is 1.0.

C. The Design of Space Decomposition and Coordination Based 3D FLC for the RTCVD System
The RTCVD system is an SDDS with three control sources, where multiple control sources have the local influence feature on the space domain.Utilizing the concept of influence  degree [25], the space domain of the wafer along the radius can be partitioned into three subregions, and then the system is decomposed into three subsystems with one distributed control source.
For each subsystem, a 3D FLC as in Appendix B is designed based on expert experience; in order to relieve the stronger couplings among subsystems, local coordination was added to the three 3D where    ,    , and    ( = , , or ) denote the gains for spatial error, error in change, and output for the th 3D FLC and   (,  = , , or ) denotes the coordination factor from the th subsystem to the th subsystem.
For detailed introduction, one can refer to [26].

Figure 1 :
Figure 1: Sketch of a catalytic packed-bed reactor.

Figure 3 :
Figure 3: Radiation flux distribution of three-zone heating lamp banks.
fuzzy sets; Δ denotes the incremental control output;   denotes a traditional fuzzy set.

Table 2 :
Performance index comparisons (SVR learning based sensing locations versus evenly distributed locations).