Applied Computational Intelligence and Soft Computing

Volume 2012 (2012), Article ID 402420, 16 pages

doi:10.1155/2012/402420

Research Article

Estimation of Fuzzy Measures Using Covariance Matrices in Gaussian Mixtures

Nishchal K. Verma

Department of Electrical Engineering, Indian Institute of Technology Kanpur, Kanpur 208016, India

Received 5 May 2011; Revised 7 February 2012; Accepted 9 February 2012

Academic Editor: Enric Trillas

Copyright © 2012 Nishchal K. Verma. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract.

This paper presents a novel computational approach for estimating fuzzy measures directly from Gaussian mixtures model (GMM). The mixture components of GMM provide the membership functions for the input-output fuzzy sets. By treating consequent part as a function of fuzzy measures, we derived its coefficients from the covariance matrices found directly from GMM and the defuzzified output constructed from both the premise and consequent parts of the nonadditive fuzzy rules that takes the form of Choquet integral. The computational burden involved with the solution of λ-measure is minimized using Q-measure. The fuzzy model whose fuzzy measures were computed using covariance matrices found in GMM has been successfully applied on two benchmark problems and one real-time electric load data of Indian utility. The performance of the resulting model for many experimental studies including the above-mentioned application is found to be better and comparable to recent available fuzzy models. The main contribution of this paper is the estimation of fuzzy measures efficiently and directly from covariance matrices found in GMM, avoiding the computational burden greatly while learning them iteratively and solving polynomial equations of order of the number of input-output variables.

1. Introduction

Generalized fuzzy model (GFM) [1–3] is the backbone of this work that employs two norms for computing the strength of a rule: the multiplicative T-norm operator for determining the strength of a rule [4–6] and the additive S-norm operator for combining the outputs of all the rules. The effect of input fuzzy sets is taken into the defuzzified output in the form of rule strengths or weights. Gan et al. [7] have simplified the formulation of GFM by setting both the inputs and the output to be jointly Gaussian and proved that the input-output relation is as an expectation using the Bayesian framework. On simplification, this turns out to be the coveted Gaussian mixture model (GMM), which is linked by an additive function between the inputs and the output. The GMM, also known as cluster-weighted modeling (CWM) [8–12], is advocated by many researchers as means of statistical modeling of input-output systems [10, 13–15]. After the establishment of equivalence between GMM and GFM in [7], the difficult-to-compute index of fuzziness of GFM has become easy-to-compute prior probability [16–23] in GMM. Further simplification of GFM will be explored when it is converted into nonadditive case.

The use of GMM in GFM has provided a generalized framework for additive fuzzy systems. A few existing additive fuzzy systems urging our attention are due to fuzzy models given in Kosko [24] and Wang and Mendel [25]. Using the framework of GMM, Abonyi et al. [26] have made a semblance of generalizing TS model [4, 27, 28] by attaching weight to each of its rules and subsequently treating it as the strength of the rule. However, nowhere they touched upon the concept of GFM that seeks to unify both Mamdani and TS models [4, 5].

Incorporation of the nonadditive property into the fuzzy sets is done so that the corresponding output of a non-additive fuzzy system is explored here. Let us throw some light on the additive and non-additive fuzzy systems [29–31]. Conventional evaluation with multiple attributes which are independent is based on the concept of additive fuzzy systems where importance of each attribute is given a weight.

Some important applications of fuzzy measures are now mentioned. A random generation of fuzzy measures have been introduced and some subfamilies of fuzzy measures tackled in [32]. A set of isometric fuzzy measures for any isometric transformation in each family of fuzzy measures (e.g., convex) is also studied. In [33] fuzzy measure is shown to be a unifying structure for modeling knowledge about an uncertain variable. Kim Le in [34] presents an expression to evaluate a fuzzy measure from set of aggregated evidences. Hierarchical autoregression model is presented in [34] using nonmonotonic fuzzy measures and the Choquet integral. Narukawa et al. in [35] discuss a space of fuzzy measures bearing the topology introduced by Choquet integral and the space of nonnegative fuzzy measure is shown to be locally convex. Fuzzy measures and integrals (e.g., Sugeno and Choquet) are used in decision making and modeling auctions [36, 37]. Fuzzy measures also find an application in [38] to track a moving object from a dynamic image sequence.

Fascinated by the ever growing importance of the fuzzy measure theory [39–45], the additive function of the GFM model in the consequent part is replaced by a nonadditive function satisfying certain axioms of the fuzzy measures. The defuzzified output of the resultant non-additive GFM is shown to be in the form of Choquet integral [41]. This formulation is intended for real-life applications in which the information from different sources needs not to be additive; hence our efforts in this work will go a long way in evolving different types of non-additive fuzzy systems, like dynamic, adaptive, and so forth, but here our attempt is only on a simple non-additive fuzzy system. To take account of the overlapping information in the fuzzy sets, Sugeno [40] has introduced the concept of -measure such that an appropriate value of can account for the interaction. In the proposed formulation, -measure takes the role of this -measure for reducing the computational complexity of the former.

Fuzzy integrals have come into vogue for the information fusion as they aggregate information from several sources. Out of all fuzzy integrals [40–44], the Choquet fuzzy integral [39, 41, 45] and Sugeno integral [46] are widely used. A handful of applications of fuzzy integrals include: handwriting recognition [47], landmine detection [48], pattern recognition [49], and decision making [37]. In the present study, the Choquet fuzzy integral is favored for it employs the non-additive property of fuzzy measures. The Choquet integral has not lost its luster even now; instead it is gaining prominence day by day and making inroads into new applications.

The main theme of the present work is the estimation of fuzzy densities, and hence fuzzy measures for the non-additive fuzzy model are derived from GFM; thus the fuzzy densities can be calculated straightaway from the covariance matrices used in GMM [50]. As the large number of input variables is increased for the purpose of fusion, the computational complexity grows exponentially [31, 51] with the -measure. To overcome this problem a new fuzzy measure known as -Measure is introduced in [52]. This new measure is just normalization of -measure, and with this modification, the fuzzy densities are decorrelated. In this is taken as a variable to be learned using the training data set. The proposed modeling technique considerably reduces the computational complexity in two steps: firstly finding fuzzy densities without learning them through iterative process and secondly using -measure that eliminates the burden of solving the exponentially increasing polynomials.

This paper is organized as follows. Section 2 briefly describes the fuzzy measures. In Section 3, the formulation for the non-additive GFM is presented. An algorithm for non-additive fuzzy modeling based on -measure is given in Section 4. Two benchmark applications and one real-time electric load demand prediction application are provided in Section 5 to illustrate the proposed methodology. Finally conclusions are drawn in Section 6.

2. Fuzzy Measures

The fuzzy measure or fuzzy capacity is a subjective evaluation introduced by Choquet in 1953 and defined by Sugeno in 1974 for fuzzy integrals. Fuzzy measure includes a number of special class of measures like Sugeno’s -measure and -measure. In this work, we make use of Fuzzy measures in quantifying the influence of the interactive inputs on the output function using the fuzzy measures. The resulting model is non-additive in the sense that the information from the interactive inputs is computed non-additively. For converting GFM into a non-additive fuzzy system, the relevant properties of the fuzzy measures [46].

Definition 1. Let us denote as a finite set with its elements sorted from and let be the power set of , that is, set of all subsets of . Then a fuzzy measure over a set is a function given by

Satisfying the two properties

(1).(2)If , and , then , which is the monotonicity relation in fuzzy measure. Next we invoke the third property, namely, the so-called -measure due to Sugeno [40] stated as follows:(3)for all , with

Example 2. Let be the input sets satisfying the properties 1 and 2; then considering as a fuzzy density satisfying the property 3, the values of fuzzy measure, can be computed recursively as

starting with the one-input set to find the first measure as

It is then combined with the next input set to compute their -measure since these two input sets are disjoint, that is, , but their fuzzy sets and are dependent, that is, . Equation (3) indicates that we must consider every time the fuzzy measure of two disjoint input sets, and , which consists of inputs . It may be noted that a single element fuzzy measure is the fuzzy density itself. The following polynomial which arises from the condition (see the appendix) needs to be solved for :

The solution of (5) is computationally intensive. We will now discuss the -measure to address this problem.

Q-Measure

The -measure is burdened with the problem of finding the roots of the polynomial equation in . As the number of dimensions, increases, the complexity of the equation increases enormously. In order to surmount this problem, -measure is proposed in [52]. It derives its name from the word “Quotient.” The -measure follows the same properties of -measure. For the finite the -measure is defined as the ratio of two fuzzy measures:

where is a function of fuzzy densities and is a polynomial in . The -measure [52] over a set is similar to in (1):

with the two properties as follows:

(1)  

(2)If , and , then , which is the monotonicity relation.

The third property is as follows:

(3) for all , with ,

Following (9), is evaluated from

The first measure is computed from

Convergence Behavior of Q-Measure

Let , Such that, , and , that is, null. Supposing and

Then the -measure of and is

Using (10) the -measure is written as

or

For , and , it is always assured that

Hence, and also . Next, we have

where . Therefore and the values lie in between 0 and 1. Hence we have

This will continue until when we will have

But ; this means that the -measure converges to unity.

3. From GMM to Nonadditive GFM with Q-Measure

The GFM [7] includes both the Mamdani and TS models. The Mamdani model (CRI-model) inhibits the property of fuzziness around the fixed centroid of the consequent part while the TS model gives a varying singleton for the consequent part in each fuzzy rule. The output of GFM is a fuzzy set, which is the output of Mamdani model with an index of fuzziness and with a varying centroid as the output of the TS model. To combine both of these properties, Azeem et al. [1, 53] have introduced a GFM fuzzy rule of the form

The fuzzy set may also be in the linguistic form with the output function which may be linear or nonlinear regression of inputs and the index of fuzziness . The fuzzy output of the th rule is . The defuzzified output obtained by applying additive S-norm to the consequent parts of GFM rules is

with . As noted in Section 1 on introduction, GFM represents the additive fuzzy system. Using the framework of GMM, we intend to generalize to the case when the input fuzzy sets have overlapping information, that is, . So, we modify the above fuzzy rule to the nonadditive GFM as

To take account of the above constraint, we form the input set from which the fuzzy measures of subsets are found recursively using (10) and (11). Using the fuzzy measures as the coefficients the output function is expressed as [31]

Equation (23) is an additive function of fuzzy measures as explained in the sequel. As the input fuzzy sets in (22) are overlapping, we take disjoint input sets from the power set and compute their fuzzy measure. The function combines the fuzzy measures from subsequent fuzzy sets, and the resulting system arising from the fuzzy rules is called nonadditive fuzzy system unlike the output function in TS model where each coefficient corresponds to its own fuzzy set and there is no concept of fuzzy measure in this model. We now impose three constraints for the model to be called non-additive:

(i)the input fuzzy sets associated with the power set are overlapping,(ii)the input sets are disjoint,(iii)the output function is additive in fuzzy measures.

We will now show that the above conditions are necessary to represent the defuzzified output of the nonadditive GFM in the Choquet integral form as follows:

where for th set of the data we have

with being the diagonal element. The firing strength of each rule is obtained by taking the multiplicative T-norm of the membership functions (by suppressing the subscripts “”) of the premise parts of the rules as . We intend to compute the coefficients of the output function by reformulating (21) in the Choquet fuzzy integral form. The role of GMM is to facilitate the computation of , and .

3.1. Choquet Fuzzy Integral and Q-Measure

When a fuzzy measure is available on a finite set , we can use a fuzzy integral as a computational scheme to integrate all values from , that is, the subsets of nonlinearly. In other words, a fuzzy integral relies on the concept of a fuzzy measure. A general definition is that given a class of functions and a class of -measures , we have a functional , which is a fuzzy integral [42]. The Choquet integral is a particular fuzzy integral [41] that serves as an alternative computational scheme for aggregating information.

Definition 3. Let -measure over a set be a function whose elements (discrete) are , then the Choquet integral of a positive function with respect to is defined by [39]

or

where . We assume here that and . To convert (24) into the Choquet fuzzy integral form, we shall represent coefficients in terms of -measures, as

Interchanging the summations in (24) leads to

Next separating out the term yields,

where is a constant. Letting

Equation (30) is simplified as

Denoting and

allows us to write (32) compactly for input variables as

or

where and for , is the monotonicity condition. Note that are the evidences provided by the input sources , respectively, and are the -measures for mth cluster. Here, we take to be zero. Equations (34) and (35) similar to (26) and (27) are the two forms of Choquet fuzzy integral [42]. We have now proved that the defuzzified output of the non-additive GFM fuzzy rules is in the form of Choquet fuzzy integral. The underlying model is nonadditive GFM because both and are nonlinear functions of . Thus the Choquet integral provides the functionality of non-additive GFM. The functionality of the Choquet integral for GMM will be discussed later. We will now estimate the unknown -measures from fuzzy densities determined using (9). We then evaluate the performance of the fuzzy model with a measure, which is a function of mean square error [54], chosen as

where is the total number of test data vectors:

where is the estimated value, is the actual output, and is the output vector.

3.2. Determination of Fuzzy Densities

The cluster-wise breakup of the estimated output of the additive fuzzy system [7] is given as

where using GMM from [7] we estimate as

Equation (38) can also be written as

where . Substituting from (39) in (40), we have

Using (32), for rules, we have the cluster-wise estimates of the output for the non-additive fuzzy model written as

From (42) it is implied that in view of (34) and (35).

Equation (42) can also be written as

or

Since (40) and (44) represent the estimated outputs for additive and non-additive fuzzy systems, respectively, and hence these must be equivalent to each other for . Equating the right hand sides of (40) and (44), we have

Thus when the fuzzy measure . As we can express the fuzzy densities in terms of the elements of covariance matrices of GMM as [7]

We are in a position to estimate and -measures and hence the coefficients using the fuzzy densities. We will then use all these in the Choquet integral to estimate the output. For convenience of notation we start with fuzzy measure of one element set and move in backward direction to find the -measure of . Since we are computing all -measures before the start of the Choquet integral, these become handy in computing the output. This is because in (32) the index is in the forward direction from 1 to .

3.3. Functionality of GMM

As we have already proved that the Choquet integral is the functionality of non-additive GFM, It is now easy to extend to this functionality the GMM case from the fact that the output is Gaussian in the nonadditive case too as per the expression

From the above it may be noted that is obtained from using GMM by the choice of in the -measure. Hence is the mean of in the non-additive case just as is the mean of in the additive case. We can conclude therefore that when is evaluated from , the Choquet integral also assures the functionality of GMM.

4. Algorithm for Estimating the Model Parameters of the Nonadditive Fuzzy Systems

The algorithm has the following steps.

Step 1. Normalize the input-output data, so that the data values lie in between 0 and 1:

where is the input-output data vector.

Step 2. Find the premise model parameters of GMM using the EM Algorithm.

Step 3. Determine fuzzy densities for each input variable in th cluster/rule using (47).

Step 4. Choose initial values of with the restriction for th cluster.

Step 5. Compute by the recursive computation involving the following equations from (10) and (11):

Step 6. Using (31) find for each input variable in the cluster/rule to obtain the matrix .

Step 7. Compute the estimated output using (34).

Step 8. Learn the model. This requires the following two subtasks:

(a) set up an objective function [54] as the case required. For batch:

for incremental,

(b) Update the values of by the Gradient-descent learning law:

where may refer to either or and is a learning factor fixed at the value 0.9 for better performance. The parameter learning can be accomplished in either of the above two ways: batch or incremental. Derivatives required in the learning process are relegated to the Appendix.

Step 9. Repeat Step 1 to Step 8, till the model parameters are within the tolerance.

Step 10. Calculate from (36).

5. Experimental Results

5.1. Gas Furnace

Here, we take up the gas furnace data [55] to demonstrate the effectiveness of the proposed algorithm. In this gas furnace, the air at a feed rate of is combined with methane to produce a mixture of gases with carbon dioxide of concentration . These are used to fit the Gaussian mixture density function with four components. The input vector after sorting is and the output is . The composition of the input vector is arrived at using the fuzzy curve approach [56]. On applying the expectation maximization (EM) [57] clustering, the number of clusters is found to be 4; hence the model will have four components. The five-variable GMM using the EM algorithm yields the parameters of GMM as shown in Table 1. These parameters are also the premise model parameters of the non-additive GFM fuzzy rules.

Table 1: Estimated parameters of A 4-component GMM using the EM algorithm.

Cluster or rule no.
Weight of the rule
Mean
Covariance

Cluster or rule no. 1

Cluster or rule no. 2

Cluster or rule no. 3

Cluster or rule no. 4

Fuzzy densities and estimated fuzzy measures of the model are given in Tables 2 and 4, respectively. The estimated values of are given in Table 3. The estimated -coefficient and -measures are given in Tables 5 and 6 respectively. Note that contains all input elements, hence its value is one. The -coefficients constitute the consequent parameters of the GMM fuzzy rules. The performance measure is found to be 0.00208. The fuzzy measures of the constituent singleton sets satisfy the monotonic relations for shown as a lattice in Figure 2.

Table 2: Fuzzy densities using GMM.

Rule

Rule no. 1
0.2473
0.7760
0.0005
0.0000

Rule no. 2
0.5699
0.8204
0.0028
0.0003

Rule no. 3
0.6284
0.8671
0.0070
0.0001

Rule no. 4
0.4844
0.7330
0.0042
0.0012

Table 3: Estimated lambda ().

Rule 1
Rule 2
Rule 3
Rule 4

−0.1241
−0.8364
−0.9108
−0.6197

Table 4: Estimated fuzzy measures.

Rule

Rule no. 1
1.0000
0.7923
0.0021
0.0009

Rule no. 2
0.9997
0.8214
0.0032
0.0003

Rule no. 3
0.9999
0.8686
0.0071
0.0001

Rule no. 4
0.9994
0.7359
0.0053
0.0012

Table 5: Estimated coefficients.

Rule

Rule no. 1
0.2077
0.7902
0.0012
0.0009

Rule no. 2
0.1783
0.8182
0.0029
0.0003

Rule no. 3
0.1313
0.8615
0.0070
0.0001

Rule no. 4
0.2635
0.7306
0.0041
0.0012

Table 6: Estimated-measures.

Rule

Rule no. 1
1.0000
0.7823
0.0013
0.0000

Rule no. 2
1.0000
0.8541
0.0635
0.0001

Rule no. 3
1.0000
0.9175
0.0863
0.0003

Rule no. 4
1.0000
0.7230
0.0169
0.0002

For the training dataset containing the first 250 data vectors, . For the remaining data vectors forming the validation data set, . This is inferior to that of the training data set, as expected. The plots of actual output and model output for both training and prediction along with the prediction error of the model output are shown in Figure 1. The model output is very close to the actual output during training, but it displays small deviations from the actual output during the prediction period. Using the premise and consequent parameters of Table 1, we have constructed the membership functions for the four inputs and one output in Figure 3 where we depict the membership function of rules-1, 2, 3, and 4 using dashed lines, dotted lines, dots with dashes lines and solid lines, respectively.

(a)

(b)

(c)

Figure 1: Plot of actual output and model output for training data set (uppermost plot) and plot of actual output and predicted output and the corresponding prediction error for Box and Jenkins’s gas furnace data.

Figure 2: Lattice of fuzzy measure values of the constituent singleton sets [39] of rule 1 for Box and Jenkins’s data.

(a)

(b)

(c)

(d)

(e)

Figure 3: (a) Membership function for input one. (b) Membership functions for input two. (c) Membership function for input three. (d) Membership function for input four. (e) Membership function for output.

The first rule consists of,

(i) the weight of the rule; 0.2991,(ii) the membership functions of the premise parts

(iii) The consequent part of a rule can be found using Table 6.

Similarly other rules can also be formed.

5.2. Industrial Dryer

In this, we consider the industrial dryer data obtained from the DaISy database. Fuel is combusted to produce the hot gas, which is blown by an exhaust fan on the raw material fed into the dryer. It is desired to determine the relationship between the functional elements of the dryer with the moisture content of the raw material at the dryer output. The measurements are designated by

(i)fuel flow rate, ,(ii)hot gas exhaust-fan speed, ,(iii)rate of flow of raw material, ,(iv)moisture content of the raw material after drying which is .

Sampling period of 10 seconds is considered to generate a total of 867 samples. Input and output data are normalized using (40) to maintain compatibility in the range 0 to 1. The input variables , , , and affect the current output . The input-output vectors are represented by . Out of these vectors 70% is taken as the training set. The remaining 30% is kept for prediction and evaluation of our proposed model.

The number of clusters is found to be 4 using EM clustering, which yields the parameters of GMM in Table 7, and these are also the premise model parameters of the non-additive GFM.

Table 7: Estimated parameters of A 4-component GMM using the EM algorithm.

Cluster or rule no.
Weight of the rule
Mean
Covariance

Cluster or rule no. 1

Cluster or rule no. 2

Cluster or rule no. 3

Cluster or rule no. 4

The estimated fuzzy densities, fuzzy measures, -coefficients and -measure of the model are given in Tables 8, 10, 11, and 12, respectively. The monotonicity relations for are also verified here for fuzzy measure values of the constituent singleton sets, shown as a lattice in Figure 5. Table 9 shows the estimated values of .

Table 8: Fuzzy densities using GMM.

Rule

Rule no. 1
0.4982
0.4295
0.3881
0.7989

Rule no. 2
0.4996
0.7368
0.6158
0.9027

Rule no. 3
0.4696
0.7543
0.3704
0.5916

Rule no. 4
0.4844
0.7330
0.0042
0.0012

Table 9: Estimated lambda ().

Rule 1
Rule 2
Rule 3
Rule 4

−0.9970
−1.0000
−1.0000
−0.9947

Table 10: Estimated -measures.

Rule

Rule no. 1
0.9669
0.9315
0.8779
0.7989

Rule no. 2
0.9027
0.8828
0.8758
0.8749

Rule no. 3
0.9500
0.6529
0.6244
0.5918

Rule no. 4
1.0000
1.0000
1.0000
0.9981

Table 11: Estimated coefficients.

Rule

Rule no. 1
0.0354
0.0536
0.0790
0.7989

Rule no. 2
0.0199
0.0070
0.0009
0.8749

Rule no. 3
0.2971
0.0285
0.0326
0.5918

Rule no. 4
0.0000
0.0000
0.0019
0.9981

Table 12: Estimated -measures.

Rule

Rule no. 1
1.0000
0.7823
0.0013
0.0000

Rule no. 2
1.0000
0.8541
0.0635
0.0001

Rule no. 3
1.0000
0.9175
0.0863
0.0003

Rule no. 4
1.0000
0.7230
0.0169
0.0002

The training data set yields for training the model. The remaining 30% data vectors constitute the validation data set. The performance measure over the prediction period is obtained as 0.0019. The results of the actual output and the model output during the training and the prediction periods are shown in Figure 4 which also includes the prediction error. The results demonstrate the effectiveness of the proposed non-additive fuzzy model using -measure. The membership functions associated with all the four rules can be obtained in the similar fashion as in Figure 3.

(a)

(b)

(c)

Figure 4: Plot of actual output and model output for training data set (uppermost plot) and plot of actual output and predicted output and the corresponding prediction error for industrial dryer data.

Figure 5: Lattice of fuzzy measure values of the constituent singleton sets of rule no.1 of Table 10 for industrial dryer’s data.

5.3. Electric Load Demand

In this, we consider the real time hourly electric load data, that is, hourly peak electric load consumption with uncertain weather information like hourly average temperature, humidity, and wind speed for the month of April 2003. The variables are designated as

(i)hourly average Temperature, ,(ii)hourly average Humidity, ,(iii)hourly average Wind speed, ,(iv)hourly peak electric load consumption which is .

A number of 720 samples of Input and output data are normalized using (40) to maintain the compatibility in the range 0 to 1. We consider 75% of the total input-output vectors of the form as the training set. The remaining 25% is reserved for the prediction. The weights before the components in have been fixed for achieving better prediction capability. It may be noted that the criteria only facilitate the selection of proper inputs by trying out different combinations. But the final curtain falls on the selection with the verification through the performance measure.

The number of clusters in this example also is found to be 4 using the EM clustering after discarding a few datasets as outliers. Application of the EM algorithm yields the parameters of GMM in Table 13. These constitute the premise parameters of the GFM. The estimated fuzzy densities, lambda values, fuzzy measure, -measure and -coefficients of the model are given in Tables 14, 15, 16, 17, and 18, respectively. The monotonicity relations can also be verified as in the previous applications

Table 13: Estimated parameters of A 4-component GMM using the EM algorithm (load).

Cluster or rule no.
Weight of the rule
Mean
Covariance

Cluster or rule no. 1

Cluster or rule no. 2

Cluster or rule no. 3

Cluster or rule no. 4

Table 14: Fuzzy densities using GMM.

Rule

Rule no. 1
0.8479
0.6488
0.5817
0.6735

Rule no. 2
0.0196
0.0633
0.0001
0.2634

Rule no. 3
0.0264
0.0100
0.0725
0.0255

Rule no. 4
0.0001
0.0469
0.0001
0.1708

Table 15: Estimated lambda ().

Rule 1
Rule 2
Rule 3
Rule 4

29.3843
−0.0776
−0.9995
0.2543

Table 16: Estimated -measures.

Rule

Rule no. 1
1.0000
0.0877
0.0439
0.6735

Rule no. 2
1.0000
0.1200
0.0569
0.2634

Rule no. 3
1.0000
0.0910
0.0818
0.0255

Rule no. 4
1.0000
0.4718
0.1974
0.1708

Table 17: Estimated -measures.

Rule

Rule no. 1
1.0000
0.0291
0.0145
0.0033

Rule no. 2
1.0000
0.1569
0.0744
0.0614

Rule no. 3
1.0000
0.1468
0.1320
0.0161

Rule no. 4
1.0000
0.3891
0.1628
0.1409

Table 18: Estimated coefficients.

Rule

Rule no. 1
0.9709
0.0146
0.0112
0.0033

Rule no. 2
0.8431
0.0825
0.0130
0.0614

Rule no. 3
0.8532
0.0148
0.1159
0.0161

Rule no. 4
0.6109
0.2263
0.0219
0.1409

The training data set is found to give . The remaining 25% data vectors constitute the validation data set. The performance measure over this set is obtained as 0.00256. The results of the actual output and the model output during the training and the prediction periods are shown in Figure 6, which also includes the prediction error. The results demonstrate the effectiveness of the proposed non-additive fuzzy model using -measure for this application too (Figure 7).

(a)

(b)

(c)

Figure 6: Plot of actual output and model output for training data set (uppermost plot) and plot of actual output and predicted output and the corresponding prediction error for hourly electric load data.

Figure 7: Lattice of -measure values of the constituent singleton sets with .

The performance measures of the applications A and B are given in Table 19 for comparison with some of the existing well-known methods of fuzzy modeling. This table shows that E of the proposed nonadditive fuzzy model is better than those of the additive fuzzy systems of Gan et al. [7] and WM method [25].

Table 19: Comparison of .

Data
WM [25] method
Gan et al. [7]
Non-additive (using -measure)
Proposed algorithm (using fuzzy measure estimation)

Box & Jenkins’s Gas Furnace data
0.0057
0.0032
0.0021
0.00208

Industrial Dryer’s data
NA
0.0027
0.0019
0.00189

6. Conclusions

This paper deals with an important issue of computing fuzzy measures directly from covariance matrices found in GMM. Use of EM algorithm in GMM provides us with the input clusters and their Gaussian memberships. The consequent parts of the fuzzy rules, that is, the output function, of GFM are altered for incorporating the non-additive property. It has to be noted that we have been still using the multiplicative T-norm in the premise part but the additive property of S-norm in the consequent part. For this the output function is defined as a linear function of fuzzy measures serving as coefficients. Computation complexity is reduced by replacing Sugeno’s -measure with the -measure in the non-additive fuzzy model.

The defuzzified output of the non-additive GFM rules is then shown to be in the form of Choquet integral. By showing the Choquet fuzzy integral as the functionality of non-additive GFM, we have enhanced its capability to tackle a variety of real-life problems. We have also proved the corresponding non-additive functionality for GMM. This has been accomplished by showing that is the mean value of . The computation of fuzzy measures required in the Choquet integral entails a lot of computational burden, as we need to solve a polynomial in . Using -measure this burden has been eliminated.

The non-additive fuzzy model is applied on the three benchmark applications, namely, Box and Jenkins’s, Dryers’ data and on the real time electric load demand data. The results demonstrate the superiority of this model over the additive fuzzy model of Gan et al. [7] and Wang and Mendel method [25]. Thus this work makes an important contribution to the field of fuzzy modeling by providing a transition from additive fuzzy systems to non-additive fuzzy systems for dealing with the real-life applications. This is done without learning the fuzzy densities and eliminating the computational complexity associated with solving the polynomial function in . In the non-additive fuzzy systems, it makes a novel contribution and will pave the way for several other applications and for further research.

Appendix

Formulation of the Learning Process

In this section we will find the derivatives required for the learning laws. To keep the thread going, we recall the relevant equations from Section 5.

For incremental learning, using (51), we obtain the derivatives in (52) as

For batch learning, using (50), (52), we obtain the derivative as

In the two applications that we dealt with, we have used the incremental learning. Now the formulation of the learning process required in the above two types of learning is presented in the following. For th set of -inputs , the estimated output using (34) is

The partial derivatives of (A.3) with respect to and fuzzy densities for required in (A.1b) and (A.2b) are obtained after rearranging as

Using (6), we obtain the derivatives as follows:

The polynomial for the general case of -inputs using (3) and (4) is

The polynomial in for is found as

The derivatives of the above with respect to are

where

The above polynomial and their derivatives in (A.8) are used in (A.4) and (A.5) to obtain a solution. Thus, we can see a considerable simplification with -measure as the solution of -polynomials is not needed. On the other hand without -measure, the derivatives of with respect to are required to learn .

References

M. F. Azeem, M. Hanmandlu, and N. Ahmad, “Structure identification of generalized adaptive neuro-fuzzy inference systems,” IEEE Transactions on Fuzzy Systems, vol. 11, no. 5, pp. 666–681, 2003.

M. F. Azeem, M. Hanmandlu, and N. Ahmad, “Generalization of adaptive neuro-fuzzy inference systems,” IEEE Transactions on Neural Networks, vol. 11, no. 6, pp. 1332–1346, 2000.

N. K. Verma and M. Hanmandlu, “Adaptive non-additive generalized fuzzy systems,” Applied Soft Computing Journal, vol. 10, no. 3, pp. 820–831, 2010.

T. Takagi and M. Sugeno, “Fuzzy identification of systems and its applications to modeling and control,” IEEE Transactions on Systems, Man and Cybernetics, vol. 15, no. 1, pp. 116–132, 1985.

E. H. Mamdani and S. Assilian, “Experiment in linguistic synthesis with a fuzzy logic controller,” International Journal of Man-Machine Studies, vol. 7, no. 1, pp. 1–13, 1975.

J. S. R. Jang and C. T. Sun, “Functional equivalence between radial basis function networks and fuzzy inference systems,” IEEE Transactions on Neural Networks, vol. 4, no. 1, pp. 156–159, 1993.

M.-T. Gan, M. Hanmandlu, and A. H. Tan, “From a gaussian mixture model to additive fuzzy systems,” IEEE Transactions on Fuzzy Systems, vol. 13, no. 3, pp. 303–316, 2005.

R. L. Flood and E. R. Carson, Dealing with Complexity: An Introduction to the Theory and Application of Systems Science, Plenum, New York, NY, USA, 2nd. edition, 1993.

G. J. McLachlan, “Cluster analysis and related techniques in medical research,” Statistical Methods in Medical Research, vol. 1, no. 1, pp. 27–48, 1992.

N. A. Gershenfeld, The Nature of Mathematical Modeling, Cambridge University Press, London, UK, 1999.

B. S. Everitt, Cluster Analysis, Edward Arnold, London, UK, 3rd. edition, 1993.

K. Y. Yeung, C. Fraley, A. Murua, A. E. Raftery, and W. L. Ruzzo, “Model-based clustering and data transformations for gene expression data,” Bioinformatics, vol. 17, no. 10, pp. 977–987, 2001.

D. V. Prokhorov, L. A. Feldkamp, and T. M. Feldkamp, “A new approach to cluster-weighted modeling,” in Proceedings of the International Joint Conference on Neural Networks (IJCNN'01), vol. 3, pp. 1669–1674, July 2001.

L. A. Feldkamp, D. V. Prokhorov, and T. M. Feldkamp, “Cluster-weighted modeling with multiclusters,” in Proceedings of the International Joint Conference on Neural Networks (IJCNN'01), vol. 3, pp. 1710–1714, July 2001.

B. Everitt, D. J. Hand, et al., Finite Mixture Distributions, Chapman and Hall, London, UK, 1981.

J. H. Wolfe, “A Monte-Carlo study of the sampling distribution of the likelihood ratio for mixtures of multinormal distributions,” Technical Bulletin STB 72-2, US Navel Personnel and Training Research Activity, San Diego, Calif, USA, 1971.

N. R. Mendell, H. C. Thode, and S. J. Finch, “The likelihood ratio test for the two-component normal mixture problem: power and sample size analysis,” Biometrics, vol. 47, no. 3, pp. 1143–1148, 1991.

G. J. McLachlan, “On bootstrapping the likelihood ratio test statistic for the number of components in a normal mixture,” Journal of the Royal Statistical Society, Series C, vol. 36, no. 3, pp. 318–324, 1987.

G. J. McLachlan, Discriminant Analysis and Statistical Pattern Recognition, Wiley-Interscience, New York, NY, USA, 1992.

R. E. Walpole, R. H. Myers, and S. L. Myers, Probability and Statistics for Engineers and Scientists, Prentice Hall, 6th. edition, 1998.

K. V. Mardia, J. T. Kent, and J. M. Bibby, Multivariate Analysis, Academic Press, 1979.

W. H. Press, S. A. Teukolsky, W. T. Vetterling, and B. P. Flannery , Numerical Recipes in C: The Art of Scientific Computing, Cambridge University Press, London, UK, 2nd. edition, 1993.

M. Aitkin, D. Anderson, and J. Hinde, “Statistical modelling of data on teaching styles,” Journal of the Royal Statistical Society A, vol. 144, no. 4, pp. 419–461, 1981.

B. Kosko, Fuzzy Engineering, Prentice Hall, London, UK, 1997.

L. X. Wang and J. M. Mendel, “Generating fuzzy rules by learning from examples,” IEEE Transactions on Systems, Man and Cybernetics, vol. 22, no. 6, pp. 1414–1427, 1992.

J. Abonyi, R. Babuška, and F. Szeifert, “Modified Gath-Geva fuzzy clustering for identification of Takagi-Sugeno fuzzy models,” IEEE Transactions on Systems, Man, and Cybernetics, Part B, vol. 32, no. 5, pp. 612–621, 2002.

M. Sugeno and T. Yasukawa, “Fuzzy-logic-based approach to qualitative modeling,” IEEE Transactions on Fuzzy Systems, vol. 1, no. 1, pp. 7–31, 1993.

M. Sugeno and G. T. Kang, “Structure identification of fuzzy model,” Fuzzy Sets and Systems, vol. 28, no. 1, pp. 15–33, 1988.

M. Hanmandlu, N. K. Verma, N. Ahmad, and S. Vasikarla, “Cluster-weighted modeling as a basis for non-additive GFM,” in Proceedings of the IEEE International Conference on Fuzzy Systems, FUZZ-IEEE, pp. 652–657, Reno, Nev, USA, May 2005.

J. R. Chang, G. H. Tzeng, C. T. Hung, and H. H. Lin, “Non-additive fuzzy regression applied to establish flexible pavement present serviceability index,” in Proceedings of the IEEE International Conference on Fuzzy Systems, pp. 1020–1025, May 2003.

N. K. Verma and M. Hanmandlu, “From a gaussian mixture model to nonadditive fuzzy systems,” IEEE Transactions on Fuzzy Systems, vol. 15, no. 5, pp. 809–827, 2007.

P. Miranda and E. F. Combarro, “On the structure of some families of fuzzy measures,” IEEE Transactions on Fuzzy Systems, vol. 15, no. 6, pp. 1068–1081, 2007.

R. R. Yager, “Uncertainty representation using fuzzy measures,” IEEE Transactions on Systems, Man, and Cybernetics, Part B, vol. 32, no. 1, pp. 13–20, 2002.

K. Le, “Fuzzy measures of aggregated evidences,” in Proceedings of the Biennial Conference of the North American Fuzzy Information Processing Society (NAFIPS '96), pp. 572–576, Berkeley, Calif, USA, June 1996.

Y. Narukawa, T. Murofushi, and M. Sugeno, “Space of fuzzy measures and convergence,” Fuzzy Sets and Systems, vol. 138, no. 3, pp. 497–506, 2003.

J.-H. Chiang, “Choquet fuzzy integral-based hierarchical networks for decision analysis,” IEEE Transactions on Fuzzy Systems, vol. 7, no. 1, pp. 63–71, 1999.

Y. Narukawa and V. Torra, “Fuzzy measures and integrals in evaluation of strategies,” Information Sciences, vol. 177, no. 21, pp. 4686–4695, 2007.

Z. Q. Liu, L. T. Bruton, J. C. Bezdek et al., “Dynamic image sequence analysis using fuzzy measures,” IEEE Transactions on Systems, Man, and Cybernetics, Part B, vol. 31, no. 4, pp. 557–572, 2001.

M. Grabisch, T. Murofushi, and M. Sugeno, Fuzzy Measures and Integrals: Theory and Applications, Physica, Heidelberg, Germany, 2000.

M. Sugeno, “Fuzzy measures and fuzzy integrals: a survey,” in Fuzzy Automata and Decision Processes, M. M. Gupta, G. N. Saridis, and B. R. Gaines, Eds., pp. 89–102, North-Holland, New York, NY, USA, 1977.

T. Murofushi and M. Sugeno, “Some quantities represented by the choquet integral,” Fuzzy Sets and Systems, vol. 56, no. 2, pp. 229–235, 1993.

L. M. De Campos, M. T. Lamata, and S. Moral, “A unified approach to define fuzzy integrals,” Fuzzy Sets and Systems, vol. 39, no. 1, pp. 75–90, 1991.

M. Friedman, M. Ma, and A. Kandel, “Numerical methods for calculating the fuzzy integral,” Fuzzy Sets and Systems, vol. 83, no. 1, pp. 57–62, 1996.

M. Sugeno and T. Murofushi, “Pseudo-additive measures and integrals,” Journal of Mathematical Analysis and Applications, vol. 122, no. 1, pp. 197–222, 1987.

Z. Wang, G. J. Klir, and W. Wang, “Monotone set functions defined by choquet integral,” Fuzzy Sets and Systems, vol. 81, no. 2, pp. 241–250, 1996.

M. Sugeno, Theory of fuzzy integrals and its applications, Ph.D. thesis, Tokyo Institute of Technology, Tokyo, Japan, 1974.

P. D. Gader, J. M. Keller, R. Krishnapuram, J. H. Chiang, and M. A. Mohamed, “Neural and fuzzy methods in handwriting recognition,” Computer, vol. 30, no. 2, pp. 79–86, 1997.

P. D. Gader, J. M. Keller, and B. N. Nelson, “Recognition technology for the detection of buried land mines,” IEEE Transactions on Fuzzy Systems, vol. 9, no. 1, pp. 31–43, 2001.

J. M. Keller, P. Gader, H. Tahani, J. H. Chiang, and M. Mohamed, “Advances in fuzzy integration for pattern recognition,” Fuzzy Sets and Systems, vol. 65, no. 2-3, pp. 273–283, 1994.

W. J. Krzanowski and F. H. C. Marriott, Multivariate Analysis Part 2: Classification, Covariance Structures and Repeated Measurements, Edward Arnold, London, UK, 1995.

A. K. Hocaoǧlu and P. Gader, “Comments on "choquet fuzzy integral-based hierarchical networks for decision analysis",” IEEE Transactions on Fuzzy Systems, vol. 7, no. 6, p. 767, 1999.

M. A. Mohamed and W. Xiao, “Q-measures: an efficient extension of the sugeno λ-measure,” IEEE Transactions on Fuzzy Systems, vol. 11, no. 3, pp. 419–426, 2003.

M. F. Azeem, M. Hanmandlu, and N. Ahmad, “Parameter determination for a generalized fuzzy model,” Soft Computing, vol. 9, no. 3, pp. 211–221, 2005.

A. R. Barron, “Predicted squared error: a criterion for automatic model selection,” in Self-Organizing Methods in Modeling, S. J. Farlow, Ed., vol. 54, pp. 87–103, Marcel Dekker, New York, NY, USA, 1984.

E. P. Box and G. M. Jenkins, Time Series Analysis: Forecasting and Control, Holden-day, San Francisco, Calif, USA, 1970.

M. F. Azeem, M. Hanmandlu, and N. Ahmad, “A new criteria for input variable identification of dynamical systems,” in Proceedings of the IEEE Region 10th International Conference on Global Connectivity in Energy, Computer, Communication and Control (TENCON '98), vol. 1, pp. 230–233, New Delhi, India, 1998.

A. P. Dempster, N. M. Laird, and D. B. Rubin, “Maximum likelihood from incomplete data via the EM algorithm,” Journal of the Royal Statistical Society B, vol. 39, no. 1, pp. 1–38, 1977.