Multimodel Representation of Complex Nonlinear Systems: A Multifaceted Approach for Real-Time Application

. Presenting an important potential in the representation of nonlinear systems, the multimodel approach remains an attractive axis for research. One of the important problems in the multimodel structure concerns the validity calculation which is a fundamental point especially when the process is corrupted with noise and/or its parameters are of high variations. A new approach based on the use of both two type of validity is proposed. A developed specification of the need of each one is explained by an optimization procedure. The conduct of this approach requires, first, the classification of the numerical data into a set of clusters. The frequency-sensitive competitive learning (FSCL) algorithm is used to select the number of models and the fuzzy k-means algorithm identify the operating clusters. From the satisfactory results in terms of precision and robustness obtained on theoretical examples, we are incited to confirm our contribution to real process reactor. The results obtained are compared to the classical approaches showing its ability to represent adequately the nonlinear process with a superior precision and accuracy and from this the classic strategy of multimodel representation is oriented towards a multifaceted approach.


Introduction
The multimodel approach has experienced a certain interest since the publication of the work of Johansen [1].The idea of the multimodel approach is to apprehend the nonlinear behavior of a system by a set of local models (linear or affine) characterizing the operation of the system in different areas of operation.The multimodel is based on the decomposition of the dynamic behavior of the system in several areas of operation.Each zone is characterized by a subsystem which contributes at the approximation of the global system.The modeling approach is interesting because it provides a simplified and transparent formulation from the nonlinear model easily to study.The most interesting idea of the multimodel approach, which will also be very useful, is its ability to approximate a nonlinear system by a weighting of local submodels via validity functions.These validity functions represent the action area of the submodel and its contribution to the global model.The validities calculation is counted among the main aspects to be considered in the multimodel approach.The validity V  of a local submodel   is a numeric value belonging to the interval [0.1].The value 1 indicates that the submodel corresponding represents perfectly the system in this region.The value of 0 implies that the submodel is completely inadequate.Thus defined, these coefficients have a great influence on the accuracy of the global model or on its control.This justifies the importance given by many researchers to the problem of validity calculation.In fact, in the literature several methods of validity estimation have been proposed.One can cite the geometric approach, the probabilistic approach, the fuzzy approaches, and the residue approach [2][3][4].Numerous validities calculation technique is proposed in literature.These techniques are classified according to the mode of creating the submodels but not in the structure repartition of different cluster forming the overall nonlinear system [5].Referring to the literature, the residue approach is the main known approach which deals with the validities' computation and usually formulated by the calculation of the geometric distance [6].A new strategy based on a fuzzy supervisor method for the estimate validities 2 Mathematical Problems in Engineering is describe in [7].A neural network is used to create the validity demonstrate a very significant results in the synthesis of the multimodel is explained in [8].Other specific method are also represented in [9][10][11][12].A great importance was considered to the work presented in Elfelly [13].In fact, a specific validity is accorded to the structure of cluster distribution.Indeed, a simple validity is accorded to the system with important variations in the same cluster and when an overlapping between clusters occurs the reinforced validity is more suitable for system when clusters presents few variations and are well separated.The latest work requires an appropriate clustering method and did not make in evidence that the process is usually noisy due to the sensors or the influence of external factors which affect the structure of clusters distribution.A new strategy of modeling is needed to quantify the limits of each submodel, referred to as model inadequacy.To formalize the selection of the appropriate submodels for the task at hand, a new method presented here is to bypass this difficulty by combining both type of validity such as simple and reinforcement for each submodel of the library.This new method is needed where are families of submodels with differing fidelity with respect to different quantities of interest.

Complex System's Modeling
2.1.Model-Base Construction Procedure.We are interested in this paper to the study of "black box" systems where we dispose only input/output measures due to the impossibility to define or to develop a mathematical model which can represent the system in its area of operation.In this case the multimodel approach is a useful and an effective tool to overcome the difficulties related to the modeling of this kind of system.The proposed approach consists on the determination of the number, structure, and parameters in order to construct the model's base.In the first step, the determination of the number of submodels will be handled by using the frequency competitive learning algorithm.In the second step, the fuzzy k-means clustering has been chosen in order to generate the different operating domains.The classification results are then used for a structure and parameters identification of the model's library.Finally a satisfactory technique is used to validate the modeling strategy by using an adequate validity computation to generate the multimodel output.

Determination of the Number of Models via FSCL.
Frequency-Sensitive Competitive Learning (FSCL) [14] is a competitive algorithm with  neuron that is trained by a dataset of  data vectors ().In the FSCL, there are three layers.The first layer distributes the input vector from the training set to the second layer where each unit   and its weight   compute the distortion between its weight vector and the input vector.In the FSCL network, each neural unit incorporates a count of the number of times it has been the winner.During the training process, all neural units are modified an approximately equal number of times.This is done by modifying the Euclidean distance used to determine the winner by an increasing function of the number of wins for each unit.After initializing randomly the weight   , for each sample  from the data  and based on minimum distance criteria the FSCL determine the winner by such that where is initial estimation of the number of clusters in the given data;  1 ≤  ≤  is the output units of dimension ;  1 ≤  ≤  is weight vectors each of dimension ;  1 ≤  ≤  is the d-dimensional input vector from dataset ;  is D-dimensional weight vector corresponding to the winner;‖ * ‖ is Euclidean distance;c is index of the unit which wins the competition;  is conscience factor used to reduce the winning rate of the frequent winners defined as follows [15]: where   refers to the cumulative number of occurrences of the node  has won the competition.After selecting the winner, FSCL updates the winner as follows: where   is the learning rate defined as follows: where    ,    , and  max are respectively the initial learning rate, the final learning rate, and the maximum number of training iterations.Convergence properties of the FSCL algorithm to local minima have been studied by Galanopoulos [16].Referring to the learning results we can easily visualize where some of the data clusters are more densely populated than others, which should ideally result to some units wining more often in those clusters than in others.

Determination of the Operating Clusters Using Fuzzy k-Means.
The fuzzy k-means is one of the most popular clustering algorithms defined by Dunn [17] and improved by Bezedek [18].Fuzzy k-means attempts to find a partition (fuzzy clusters) by the calculation of cluster centers and the assignment of a set of data points   to these centers based on the minimization of an objective as where  is degree of membership of   to cluster , ∑  =1   = 1; is number of observation; is number of cluster;  is th data point;  is center of cluster .The algorithm consists of the following steps: (1) Initialize the membership matrix  = [  ] with random value between 0 and 1.
(2) Calculate fuzzy centers using (3) Compute a new matrix U using (4) The iteration will stop if where  is a termination criterion between 0 and 1.

Structure and Parametric Identification of the Base-Models.
Dealing with the linear models, for each dataset related to a cluster , a structural and parametric identification must be carried out.The structure of each model is given by where   and   are the parameters of the  th model.Structural identification relative to the estimation of the order d c of each model of the library uses the instrumental determinants' ratio-test [19].This method consists in construction the following matrix   named as the information matrix and giving by where   is the observations' number.The instrumental determinants' ratio RDI(d) is given by the following relation: For each value of , we compute the two matrixes   and  +1 furthermore the ratio given by ( 12) is estimated.The reserved value of d is the value where the matrix  +1 is singular; hence the RDI quickly increases and this is for the first time.The parametric estimation using the Recursive Least Square technique is maintained for the diverse obtained vector of each cluster.

Computation of Validities
The validities' analysis is among the main issues to consider in the multimodel approach.The validity of a model evaluates the contribution of the model to describe the system behavior in its full range and it has a great influence in the precision of the global model as well as in the performance of the global control law [6].The residue approach is the most technique cited in the literature which deals with the computation of validity.As formulated by the geometric distance, the residue expresses in each instant the difference between the systems' output and the different model output and is indicated by the following formula: where  is number of model in the base, is the process output,  is output of the submodel   .

Simple Validity Calculation. The simple validity V
is given by where = 1 and the normalized formula of the residue    is expressed by

Reinforced Validity Calculation.
The reinforced validity is defined by where

Multifaceted Validity Computation Approach.
A comparative study between the two considered validities is made in [20].The selection of the appropriate validity depends on clustering results, i.e., the clusters structure and repartition.The simple validity is suitable for systems whose dynamics are described by operating domains having one or more overlapping zones.On the other hand, it becomes less efficient and can induce disturbance phenomena in the case where the operating classes are well separated.In this case the choice of a reinforced validity is more efficient.A change in system parameters or external perturbations may affect and change the structure of different classes.These changes of cluster repartition conduct to a multifaceted comportment of the process.A simple validity may substitute for a reinforced validity and vice versa in order to try to find an adequate behavior of the model system.How can we profit from the performances introduced by each type of validities?The choice of one or the other at each instant can be made via an optimization procedure, where the impact of the validity is independent of the dynamic system and/or the structure repartition of the clusters.The multifaceted nature of the changes cluster repartition which affects the accuracy of modeling can be done by the use of a multifaced validity computation.In the proposed approach, a multifaceted approach of the validity computation is expressed by (17) where   and   are scalar parameters witch can be determined by minimizing the following expression: −  ()) ,  = 1, . . .,   (18) with The parameters of ( 18),   and   , can be determined by applying a standard-least square algorithm [21].Introduce the following regressor vector: And the parameters vector is as follows: The recursive least squares estimate of  is where  denotes the covariance matrix.The multimodel output   is calculated by a fusion of the model's outputs   weighted by their respective multifaceted validity indexes V   as illustrated by the following expression:

Model Validity Test
In order to evaluate the performance of the proposed approach, three different performance criteria are considered.
(i) The normalized root mean squared error (NRMSE) is used and given by is the process output and ŷ is the estimated process output.
(ii) The absolute errors   ,   , and   are defined by

Simulation Examples
Numerical simulations are first presented to compare the proposed approach to the conventional multimodel design and investigate its impact to model highly nonlinear systems.
In the second step the proposed approach is applied for real process reactor and two academic examples of nonlinear system: a biological reactor and a liquid level control.

Deterministic Case.
The first example is a complex discrete system whose evolution is given by [7]  () = − 1 ()  ( − 1) −  2 ()  ( − 2) The model given by ( 31) is used to generate output data for a random input distribution from -1 to 2. The adequate number of clusters is determined using the FSCL algorithm.Figure 1(a) gives the results.With five neurons used in the output layer two centers move away from the observation data.We can conclude that the number of cluster is equal to three.The classification results are also given in Figure 1(b).
From each of the three datasets relative to the various clusters, the orders and the parameters of the transfer functions relative to the three base-models are estimated.The order of each model using the RDI method is equal to two and the recursive least squares methods gives the different transfer functions.
In order to validate the proposed process modeling, the following input sequence is considered: The simulation results of the multimodel output are given in Figure 2 where the multimodel output is given by (25).
Based on clustering results, the reinforced validities are more convenient.A comparison study is that made on between the novel approach and the conventional one based on reinforced and simple validity computation.So, we have illustrated in Figure 3   ,   , and   variations.The results demonstrate that the novel approach tracks the real output with a very small error.This consequence is obtained by the adequate way of validity choice approved in the optimization phase at each instant .In fact, we can exploit at every moment whose validity is necessary to satisfy good results with minimal error.In Figure 4 we have plotted each validity calculation and their evolution after the optimization procedure.So, we can see in the case study the following: (i) Before the optimization procedure the reinforced validity for submodels 1, 2, and 3 (dotted line) is more convenient than those of simple validity; this is obvious in view that the cluster repartition presents few variations and is well separated (Figure 1(b)).So the modeling is more accurate with this kind of validity as illustrated in Figure 3.
(ii) After the optimization procedure we can see that on the one hand the simple and reinforced validities for submodel 1 are more convenient than those of submodels 2 and 3. On the other hand, the reinforced validity calculations (dashed line) are more suitable for more submodel 2 and submodel 3. The multimodel output   is based on all the modified simple or reinforced validities.

Stochastic Case.
A white noise signal with normal distribution is added to the system's output given by (31).The signal to noise ratio equals 15  with a variance  = 0.6394.Figure 5 records the evolution of the classification data.The application of FSCL algorithm leads also to three different clusters.The true and the simulated output are shown in Figure 6 and the differences in the performances are carried out in Figure 7.This result shows that the multimodel using both two types of validities is more accurate than using simple or the reinforced validities.
With the same validation sequence given by (33), the results of simulation are plotted in Figure 8.The numerical values of the normalized root mean squared error are, respectively, NRMSE = 0.4630 for the case with simple validity, NRMSE= 0.3688 using the reinforced validity, and NRMSE = 3.250210 −004 for the proposed approach.On the other hand, the variations of the   ,   , and   are carried out in Figure 9 showing that the latter has the smallest error.

Interpretations. Based on the comparative study done in
Elfelly [20], the following can be noticed:  In this case the reinforced validity compared to the simple validity is better-adapted.On the other hand, the evolution of the   and   in Figure 3 proved that the proposed approach shows the few modeling errors.
(ii) In the noise case, the structure of clusters changes and the clusters present several variations in the same cluster (Figure 10(a)).So, compared to the reinforced validity the simple validity is more convenient.However, for the case study the results presented in Figure 8 proved that the proposed approach is more suitable.
(iii) Then again when the process is of a high parameter's variations the structure repartition of the clusters changes as presented in (Figure 10(b)).The use of simple or reinforced validity does not give better results compared to the novel approach.As presented in Figure 9 the evolution of the   is more suitable than   or   .
In the context of our approach, we do not seek to study the structure distribution of clusters, but we are interested in the use of both two types of validity weighted each one at every instant  by a certain degree   or   .We can conclude that, for the proposed approach, the validity computation is    independent of the clustering results and the multifaceted approach allows a good modeling in the three different cases.
In future work, we will study the influence of the clustering algorithm such as the Kohonen network or the k-means algorithms.

Process Reactor.
The process studied in (Figure 11) consists of a 2-l cylindrical stainless-steel reactor installed in the laboratory of process control at the Engineers School of Gabes (Tunisia).Its temperature is regulated by means of a fluid circulating through a surrounding jacket.The circulating fluid is assured by the pump (P2).Depending on whether the reactor temperature has to be raised or lowered, the fluid is either heated by a heating changer (E1) with a set of three resistors whose electric power can be varied from 0 to 3 KW or cooled in a tubular cooler (E2) whose cooling rate is changed by varying the external cold water flow from 0 to 1200 lh −1 .A vertical stirrer (S), set along the reactor, is continuously rotated at a constant speed (usually 600 rpm) in order to keep the medium as homogeneous as possible in both temperature and composition.Should a semibatch recipe be applied, a specific device (R1), located above the reactor, allows to supply using pump (P1) any additional reactant.The reaction carried in this semibatch reactor is an esterification reaction, which is given by the following scheme: For identification experiments, we have used the following applied procedure: at the beginning, an initial volume of 2-l of the mixing acid and alcohol was put in the reactor at the room temperature.Then the reaction was heated as quickly as possible with the maximum power (3 KW) up to a temperature of 110 ∘ C, that is to say, close to desired temperature.It has been proved in previous works [22,23] that this value of the reactor temperature corresponds to the running point of the process where the efficiency of the reaction is optimal.We have plotted in Figure 12 the collect data used for system identification.
Based on the FSCL algorithm we have considered five neurons in the output layer, so two centers move away from the observation data as illustrate in Figure 13.We can conclude that the number of clusters is equal to three.The obtained data relative to each cluster is exploited for both structural and parametric estimation of base-models where the order of each model using the RDI method is estimated to two.The fusion of each model by the new technique of validity computation leads to the results given by Figure 14.
In the free case noise the relative errors evolution is given by Figure 15 which shows that the proposed multimodel concept surpasses in accuracy the classical paradigm of modeling using one validity computation.With an additional noise of 38db it was noticed via the relative errors given in Figure 16 that there is a change in the clusters structure repartition.In fact, in this situation the simple validity is more convenient.We can conclude as it is already mentioned before that cluster repartition balances between two faces.We have plotted in Figure 17 the validity action area repartition of each validity in terms of SNR (dB).The validity takes the value 1 when is considered and the value 0 if not.The choice of multifaceted action validity is better in all studied cases.

Biological Reactor.
In order to highlight the interest and contributions of the new modeling aspect and based on the same system that is the bioreactor, we compare our results with those given by [13,24].Being a good academic example of nonlinear system, the biological reactor has been treated in some works for the purpose of illustration in different approaches of modeling and controlling nonlinear systems [25][26][27].The process is therefore a bioreactor in which the microorganisms develop by consuming a substrate.The reactor, supposed to be infinitely mixed and of constant volume V, is supplied with substrate (constant concentration) with a volumetric flow rate F. These two components are assumed to be present at low concentrations.Denoting by  1 and  2 , respectively, the concentrations of the microorganisms and of the substrate and assuming that the specific rate of growth is given by the Contois model [28] we obtain the following discrete model: +1 =  (2)   − 0.5  (1)    (2)     (1)    (2)    − 0.5   (2)   + 0.05  ,   =  (1)   , The output of the system  is the concentration of microorganisms and the control input denoted  is the output flow rate.The use of multimodel in this new configuration is considered as our task and validates the acquired model.The purpose of the model is the simulation of the bioreactor.Two separate data in the experiment design are used.The first is 2416 points collected for the identification of the number of models and the structure of each submodel.In this context the system described by (36) is exited with a signal  in the form of 4-seconds-long with random amplitude between 0 ≤  ≤ 0.7.The second type of data is a 602 training point used to validate the modeling strategy.The FSCL algorithm is used to search the adequate number of clusters.With the following parameters of    = 0.45 and    = 0.041 and considering a 500 training iteration of the 2416 data by the use of e of 15 neurons, in Figure 18 six clusters centers move away from the data which results that the number of cluster is equal to nine.
Thus with only 9 linear models against 10 obtained by Elfelly [27] and against 196 models obtained by the modeling application is proposed in [24].We have succeeded in designing a new multimodel structure for the system whose performance is better to those modeling proposed in [24,27].This difference in size of the database helps to highlight our approach which guarantees satisfactory representativeness with a smaller number of models compared to the same studied system.From the results given by the FSCL algorithm, the multimodel is constructed based on 9 submodel extracted from each cluster with an order equal to two for each one.The proposed modeling approach presents the best result with NRMSE of 3.1774.10−4 compared to the results given with a reinforced validity which presents NRMSE of 0.0382 [27].In a comparative study the evolution of the novel approach is exposed in Figure 19 and the case where we used a reinforced validity is visualized in Figure 20.The evolution of the relative errors given in Figure 21 shows that the proposed multimodel concept surpasses in accuracy the modeling paradigm using the classical multimodel approach.The output   follows the real output with a negligible error.This error is more important when applying the reinforced validities (e r renf ) or the simple validities method (e r simpl ).When the process is corrupted by an additional noise (the process has noise whose level is 25 dB of SNR), the modeling results confirm the performance of the proposed approach.Indeed, the novel approach presents a NRMSE of 0.0038 against a NRMSE of 0.0803 for the case with reinforced validity.Figure 21 presents the evolution of the relative error between real and multimodel outputs, so the accuracy of modeling is more convenient with the novel approach than those using only one validity.It should be mentioned that the proposed strategy has hence more noiseimmunity feature than the classical approach.In fact, the relative error (e r mul ) does not exceed 10% in both cases of Figures 21 and 22.

Liquid Level Process.
The following example is used to illustrate the advantages of the proposed identification method, using a laboratory process which consists of two cascades tanks as studied in Abonyi [29].The identification dataset contains 1400 samples, using an input signal of the form given in Figure 23 The application of the FSCL algorithm is thus highlighted on the set of data collected from the process in order to determine the number of models of the library.Using seven neurons, after 500 iterations of the 1400 datasets, the algorithm leads to a concentration of 4 clusters centers and 3 centers have moved away from the dataset, which leads to the  conclusion that the process can be modeled by 4 submodels against 6 submodels in Abonyi [29].The plotted repartition centers are made in Figure 24.
The RDI value for each cluster leads to a value 2 and the least square estimation is used to develop the different submodels of the library.Moreover, a comparative study is highlighted for the multimodel approach using, respectively, a simple validity, a reinforced validity, and the envisaged approach.The simulation results are plotted in Figure 25, showing that the tracking accuracy is better in the multivalidity approach.This is confirmed by inspecting the value of NRMSE which is, respectively, of 0.0682 for the case with simple validity, a value of 0.0177 for the case using a reinforced validity, and a value of 0.0019 for our approach.Figure 26 presents the relative errors evolution showing that the output y mul follows the real output with a negligible error.This error is more important when applying the reinforced validities (e r renf ) or the simple validities method (e r simpl ).
For the robustness study with respect to a noise, the process is tested with an additional noise of level 18dB.In this case, we have limited ourselves to the representation of the absolute value of error for each case (Figure 27) where y and y mul are used in percentage of their full range.The noiseimmunity is clear for the proposed approach.

Conclusions
A new form of validity calculation has been developed.The multifaceted validity procedure relies on the use of a convex combination for simple and reinforced validity.The proposed design procedure has been applied to numerical examples and academic benchmark such as biological reactor and liquid level control shows its effectiveness compared to conventional approaches.Independently of the clusters repartition, the envisaged approach always guarantees the best results of modeling.The novel approach which is also tested for a real process reactor presents better results.

Figure 4 :
Figure 4: Evolution of simple and reinforced validity for each submodel (dotted line: original-dashed line modified).

Figure 5 :Figure 6 :
Figure 5: The evolution of the classification data (noise case).

( i )
For the deterministic case, the clusters present very few variations and are well separated (Figure 1(b)).

Figure 9 :
Figure 9: Absolute error between real and multimodel outputs.
Cluster repartition high parameter's variations

Figure 17 :Figure 18 :
Figure 17: Evolutions of the validity action area.
(a); the measured process output is plotted in Figure 23(b).Input and output signals are plotted in percentage of their full range.