Interest Measures for Fuzzy Association Rules Based on Expectations of Independence

the original


Introduction
Searching for association rules is a broadly discussed, developed, and accepted data mining technique [1,2].An association rule is an expression  ⇀ , where antecedent  and consequent  are conditions, the former usually in the form of elementary conjunction and the latter being usually atomic.Such rules are usually interpreted as the following implicational statement: "if  is satisfied then  is true very often too." Naturally, analysts are interested only in such rules that are somehow interesting, unusual, or exceptional.To assess rule interestingness objectively, there have been developed many measures of rule interest or intensity.Among the most essential, support and confidence are traditionally considered.An objective of association rules mining is to find rules with support and confidence above some user-defined thresholds.
Searching for association rules fits particularly well on binary or categorical data and many have been written on that topic [1][2][3][4].For association analysis on numeric data, a prior discretization is proposed, for example, by Srikant and Agrawal [5].Another alternative is to take an advantage of fuzzy logic [6].
The use of fuzzy logic in connection with association rules has been motivated by many authors (see e.g., [7] for recent overview).Fuzzy association rules are appealing also because of the use of vague linguistic terms such as "small" and "very big" [8][9][10][11].
In this paper, we focus on three measures of rule intensity that are all based on comparison between the observed support and the support that is expected under the assumption of independence of the rule's antecedent and consequent.These measures are lift, leverage, and conviction.All of them were initially developed for nonfuzzy (i.e., "crisp") association rules.
Lift was firstly described in [12] under its original name "interest." It was well studied for association rules on binary data in [13,14].Lift is defined as a ratio of observed support  ∧  to the support that is expected under the assumption of independence of  and .
On the other hand, leverage [15] measures the difference of observed and expected supports.(Hence, it is very similar to lift.) Conviction [12] is defined as a ratio of expected and observed support of ∧¬.Although it is very similar to lift, its properties are more similar to confidence.Lallich et al. [16] provide a nice overview of many other crisp rule measures.
As quite many have been written about these measures for crisp association rules, not so much has been done with respect to fuzzy rules.Unfortunately, simplicity of definitions for crisp rules sometimes led to oversimplified definitions for fuzzy rules.Some authors believe the generalization of 2 Advances in Fuzzy Systems those crisp measures for fuzzy data is as trivial as substituting crisp terms with analogous fuzzy terminology inside of crispcase definitions; see, for example, [17,18].Unfortunately, as discussed in this paper, such oversimplification may lead to erroneous outputs.In order to preserve some nice mathematical properties, one must take care of the type of the -norms being used.
In Section 2, a brief theoretical background for both binary and fuzzy association rules is provided.Section 2.3 discusses shortly naive definitions of lift, leverage, and conviction and shows a simple example where the rational interpretations of these measures become broken.Before providing corrected definitions in Section 4, an essential notion of expected support is analysed in Section 3. Finally, Section 5 concludes the paper with summarization of the achieved goals and drawings of the possible directions of future research.
If O is a random sample, then observing  ∈ O is a random event.Then also a random event X may be defined on the basis of the truth value of the predicate () and support supp() becomes an estimate of a probability (X).Then also supp(¬) would correspond to the probability (X), where X is complementary event to X. Confidence conf( ⇀ ) would then be an estimate of conditional probability (Y | X).
Hence, lift lift( ⇀ ) is a ratio of observed support to the support that is expected under the assumption of independence, leverage lever( ⇀ ) is a difference of observed and expected support, and conviction is a ratio of the expected support of  appearing without  to the observed support supp( ⇀ ¬).

Naive Definition of Lift, Leverage, and Conviction for Fuzzy Association Rules.
A naive approach for introducing lift, leverage, and conviction into the fuzzy association rules framework is to use simply their definitions (7), (8), and (9) for binary rules and replace binary support (4) and ( 5) and confidence (6) with their fuzzy alternatives (12) as, for example, in [17,18].Unfortunately, that approach works well only for ⊗ being the product -norm.As indicated in the following experiment, using minimum or Łukasiewicz norms may lead to erroneous interpretations.
Experiment 1. Two vectors  and  of size  = 2000 were randomly generated from the uniform distribution on the interval [0, 1] so that they are stochastically independent.Above-described naive versions of lift, leverage, and conviction of a rule  ⇀  were computed with using minimum, product, and Łukasiewicz -norms as ⊗; see Table 1 for results.As discussed in Section 2.1, stochastically independent data are expected to result in lift and conviction being close to 1 while leverage is expected to be close to 0. As can be seen, this is the case only for product -norm.
The values of naive lift and naive leverage wrongly indicate positive (resp., negative) relationship, if the minimum (resp., Łukasiewicz) -norm is used.Paradoxically, naive conviction indicates opposite sign of relationship.For the sake of simplicity, let us assume  and  are sets containing a single fuzzy attribute; that is, || = || = 1.For more complex cases, a new attribute can be created from the set of fuzzy attributes by using (10).
If O is a set of randomly selected objects, one can consider membership values () and () as random variables X and Y and treat the independence of fuzzy attributes as stochastic independence of random variables X and Y.
Two random variables X, Y are stochastically independent, if the combined random variable (X, Y) has a joint probability density as If X and Y are two independent random variables from interval [0, 1], then is a random variable with probability density function   (, ) =  X,Y (, ).By definition, expected value [Z] of a random variable Z is a weighted average of all possible values.More formally, where  Z is a probability density function of random variable Z.
Similarly, an expected value [(, )] is a weighted average of all possible (, ) pairs, namely, In real setting,   (, ) is unknown but we can estimate its values from data (i.e., from objects O and their fuzzy attributes A) by using the assumption of independence (14) as follows: where count  () is the number of objects from O that belong to  ∈ A with degree ; that is, count Assuming  ∈ {() |  ∈ O} and  ∈ {() |  ∈ O}, we obtain, from ( 15), (17), and (18), Since X and Y are independent, [fsupp( ⇀ )] =  ⋅ [(, )] and hence expected value of fsupp( ⇀ ) is Now, we are ready to define notions of expected support and expected confidence.
Proposition 5. Let ,  be sets of fuzzy attributes and let ⊗ := ⊗  ; that is, the product -norm is being used as a conjunction.Then, Proof.= fsupp () ⋅ fsupp () . ( The second inequality follows directly from Proposition 3.
The second inequality follows from ⊗ Łuk (, ) ≤  ⋅  because then we have

Correct Definition of Lift, Leverage, and Conviction for Fuzzy Association Rules
Now, we are ready to provide a correct definition of lift, leverage, and conviction in the framework of fuzzy association rules.
Proof.(1) and (2) directly follow from the definitions and from the fact that -norms are commutative.
(1) and (2) directly follow from the definitions and from the fact that -norms are commutative.
Corollary 12 copies properties that are well known for crisp variants of lift, leverage, and conviction.In practice, the use of these equations is much more convenient than the original ones from Definition 8.However, note that Corollary 12 holds only if product -norm ⊗ prod is used.For other -norms such as minimum ⊗ min or Łukasiewicz ⊗ Łuk , Definition 8 must not be oversimplified that way.See the subsequent corollaries for more details.
Proof.Everything directly follows from Definitions 2 and 8 and from Propositions 6 and 11.

Conclusion
Lift is a ratio of observed support (resp., confidence) to the support (resp., confidence) that is expected under the assumption of independence.Leverage is similar to lift, since it is a difference between observed and expected support.Conviction is often treated as an alternative to confidence; nevertheless, it is defined on the basis of observed and expected support too.
In this paper, a correct definition of lift, leverage, and conviction for fuzzy data was provided.It should be stressed here that there already exist some research papers that use incorrect (a.k.a."naive") definition of fuzzy lift (e.g., [18]) that is also discussed here.
The naive definition does not preserve interpretation of positive/negative relationship.For crisp lift and conviction (resp., leverage), the stochastically independent attributes  and  yield lift( ⇀ ) ≈ 1, conv( ⇀ ) ≈ 1, and lever( ⇀ ) ≈ 0. As shown in Experiment 1, this is no more the case for naive definitions of these measures.On the other hand, Experiment 2 shows that correct definitions of lift, leverage, and conviction again recover that feature for fuzzy data.
All the lift, leverage, and conviction definitions are similar to their "crisp" alternatives (i.e., definitions for binary data) if the -norm being used is the product ⊗ prod .For Łukasiewicz ⊗ Łuk and minimum ⊗ min -norms, a more complicated computation takes place.
In [20], an algorithm was developed in for fast evaluation of fuzzy lift.A future research will therefore address improvements of association rules search algorithms by introducing fast computations of other measures, adding pruning heuristics based on boundary conditions provided by Corollaries 13 and 14.Also, other interest measures may be studied and their applicability to fuzzy rules may be considered.

Table 1 :
Comparison of lift, leverage, and conviction computed with different -norms on stochastically independent data generated randomly from uniform distribution.

Expected Support of a Conjunction of Fuzzy Attributes under the Assumption of Independence
As indicated in Experiment 1 presented in Section 2.3 above, naive lift, leverage, and conviction no more behave like their alternatives for crisp data: they no more represent a ratio of what is observed to what is expected under the assumption of independence.To recover their definitions, independency of fuzzy attributes must be treated correctly.Only then a proper definitions of lift, leverage, and conviction can be formulated.Given sets  and  of fuzzy attributes, what support of  ∧  is expected if  and  are independent?Moreover, what does independency of fuzzy attributes mean?

Table 2 :
Comparison of lift, leverage, and conviction computed with different -norms on stochastically independent data generated randomly from uniform distribution.

Table 2 .
Since the data are randomly generated from uniform distribution, they are stochastically independent; hence, lift and conviction should be close to 1 and lift should be close to 0 regardless of -norm being used.As one can see, the results in Table2are perfectly in compliance with our expectations.