A New Uncertainty Evaluation Method and Its Application in Evaluating Software Quality

Uncertainty theory is a branch of axiomatic mathematics dealing with experts’ belief degree. Considering the uncertainty with experts’ belief degree in the evaluation system and the different roles which different indices play in evaluating the overall goal with a hierarchical structure, a new comprehensive evaluation method is constructed based on uncertainty theory. First, index scores and weights of indices are described by uncertain variables and evaluation grades are described by uncertain sets. Second, weights of indices with respect to the overall goal are introduced. Third, a new uncertainty comprehensive evaluation method is constructed and proved to be a generalization of the weighted average method. Finally, an application is developed in evaluating software quality, which shows the effectiveness of the new method.


Introduction
Due to human language and technology difficulties, it is difficult to provide an effect and objective evaluation for a system.Therefore, many scholars attempt to establish new mathematical methods to make evaluation results consistent with actual situations.Accordingly, Saaty [1] proposed the analytic hierarchy process (AHP), a method to address multicriteria decision analysis with quantitative and qualitative information.In AHP, human judgments are represented as exact numbers.However, some decision-makers may be reluctant or unable to assign exact numbers to comparison judgments because some evaluation criteria are subjective and qualitative.Therefore, Wu and Tsai [2] used both AHP and decisionmaking trial and evaluation laboratory methods to evaluate the criteria in autospare parts industry in Taiwan.Kumar et al. [3] presented a new general procedure to construct the membership and nonmembership functions of the fuzzy reliability using time-dependent intuitionistic fuzzy set.By using the finite Markov chain imbedding approach, Zhao and Cui [4] presented a unified formula with the product of matrices for evaluating the system state distribution for generalized multistate -out-of-:  systems.Chen et al. [5] provided an evaluation method for enterprisers making investment decisions under hybrid cloud environment using grey system theory.Lee et al. [6] proposed a systematic approach to evaluation of new service concepts by integrating the merit of group analytic hierarchy process in modeling multicriteria decision-making problems.Geng et al. [7] presented a new integrated design concept evaluation approach based on vague sets in order to provide a method for complicated multicriteria decision-making problem under uncertain environments.Aiming to evaluate the government and the monopolist about the consumer's taste, literature [8] was devoted to the characterization and quantitative representation of imprecise and vague uncertainties and measures of information produced by sources of the considered type.Kramosil [9] introduced the possibilistic variants of both the minimax (the worst case) and the Bayesian optimization principles and applied them in decision-making under uncertainty processed.Using finite-time control and backstepping control approaches, Li et al. [10] proposed a new robust adaptive synchronization scheme to make the synchronization errors of the systems with parameter 2 Journal of Applied Mathematics uncertainties zero in a finite time.Lan et al. [11] presented a bilevel fuzzy principal-agent model for optimal nonlinear taxation problems with asymmetric information and so on.
The above methods address imprecise information, such as human language or experts' degree of belief using fuzzy set theory (see Zadeh [12]), vague set theory, and grey system theory.However, for the evaluation system, the observed data are often not adequate and we have no choice but to invite some domain experts to evaluate the belief degree that an index belongs to an evaluation grade.In this situation, many surveys show that this imprecise information behaves like neither fuzziness nor randomness (see Liu [13] and Liu [14]).And it was showed by Kahneman and Tversky [15] that human beings usually overweight unlikely events.This fact makes the personal belief degree have much larger variance than the frequency.
In this case, Liu [13] proposed uncertainty theory to deal with belief degree, and Liu [14] refined uncertainty theory.Nowadays, uncertainty theory has become a branch of axiomatic mathematics.The first fundamental concept in uncertainty theory is the uncertain measure, used to measure the degree of belief in an event.The second concept is the uncertain variable, used to represent quantities with imprecise information (e.g., the exact value of oil field reserve).The third concept is uncertainty distribution, which is used to describe uncertain variables.Uncertainty theory has been applied to many areas.Liu [16] established a theory and practice of uncertain programming, Liu [17] applied uncertainty theory to risk analysis and reliability analysis, Liu [18] studied hybrid logic and uncertain logic, Liu [19] proposed inference rule with applications to uncertain control, and Liu [20] studied uncertain process with applications to inference risk model.To explore the recent developments in uncertainty theory, readers may consult Liu [21].
Subsequently, Liu [22] described the weights of indices and the score values of indices with uncertain variables and proposed a comprehensive evaluation method based on uncertainty theory.However, in some certain kinds of assessment domains, we find that different bottom indices play different roles in the evaluation of the overall goal.For example, suppose that  1 and  2 are two students whose four features are shown in Table 1.
Therefore, the feature vectors of  1 and  2 are  1 = (1, 20, 180, 80) and  2 = (1, 20, 170, 75).Because  1 and  2 are with the same "Gender" and "Age, " we cannot identify them from the two features.In other words, "Gender" and "Age" do not take effect at all in the identification.Furthermore, because the "Body Height" of  1 and  2 is 180 and 170, respectively, they can be identified by "Body Height." Of course, they can also be identified by "Body Weight." In a word, the four features play different roles in the identification of  1 and  2 .For this reason, a weight for each bottom index with respect to the overall goal is introduced to show the different roles of different bottom indices.And without loss of generality, when different bottom indices play the same role in other assessment domains, weights of indices with respect to the overall goal are equal.Considering these reasons, a new evaluation method is proposed based on uncertainty theory.The structure of this paper is as follows.Section 2 provides some relevant concepts about uncertainty theory.Section 3 establishes a new uncertainty evaluation method based on uncertain sets and uncertain variables.Section 4 gives an application of the proposed method in evaluating software quality, and the conclusions are presented in Section 5.

Preliminaries
In this section, we provide some useful definitions of uncertainty theory.
Let Γ be a nonempty set, let L be a -algebra on Γ, and let  be a set of real numbers.Each element Λ ∈ L is called an event.The uncertain measure M, defined on L, was proposed by Liu [13] as follows.
Definition 1 (see Liu [13]).The set function M is called an uncertain measure if it satisfies the following.
Axiom 3 (subadditivity axiom).For every countable sequence of events {Λ  }, we have The triplet (Γ, L, M) is called an uncertainty space.Let (Γ  , L  , M  ) be uncertainty spaces for  = 1, 2, . ... Write Then the product uncertain measure M on the product algebra L is defined by the following product axiom [23].
Remark 2. Uncertain measure is interpreted as the personal belief degree (not frequency) of an uncertain event that may occur.It depends on the personal knowledge concerning the event.The uncertain measure will change if the state of knowledge changes.Definition 3 (see Liu [13]).An uncertain variable is a measurable function  from an uncertainty space (Γ, L, M) to the set of real numbers.That is, for any Borel set  of real numbers, the set is an event.
Definition 6 (see Liu [14]).An uncertain variable  is called zigzag if it has a zigzag uncertainty distribution denoted by (, , ), where , ,  are real numbers with  <  < .
Then the expected value of  is defined by (11) provided that at least one of the two integrals is finite.
Remark 10.Expected value is the average value of uncertain variable in the sense of uncertain measure and represents the size of uncertain variable.
Example 11 (see Liu [21]).The zigzag uncertain variable  ∼ (, , ) has an expected value Theorem 12 (see Liu [14]).Let  and  be independent uncertain variables with finite expected values.Then for any real numbers  and , one has An uncertain set is a set-valued function on an uncertainty space that attempts to model "unsharp concepts, " which are essentially sets but their boundaries are not sharply described (because of the ambiguity of human language), such as "young" and "tall." A formal definition is given as follows.
Definition 13 (see Liu [19]).An uncertain set is a measurable function  from an uncertainty space (Γ, L, M) to a collection of sets of real numbers.For any Borel set  of real numbers, that is, for any Borel set  of real numbers, both of are events.
Definition 14 (see Liu [24]).An uncertain set  is said to have a membership function  if for any Borel set  of real numbers one has The above equations will be called measure inversion formulas.Remark 15.Let R be a set of real numbers.When an uncertain set has a membership function  on R, we immediately have Liu [24] proved that a real-valued function  is a membership function if and only if 0 ≤ () ≤ 1 (see Figure 2).
Example 16 (see Liu [21]).By a triangular uncertain set we mean the uncertain set fully determined by the triplet (, , ) of crisp numbers with  <  < , whose membership function is Definition 17 (see Liu [24]).Let  be an uncertain set with a membership function .Then the set-valued function is called the inverse membership function of .
Theorem 19 (see Liu [24]).Let  and  be independent uncertain sets with membership functions  and ], respectively.Then their union  ∪  has a membership function Theorem 20 (see Liu [24]).Let  and  be independent uncertain sets with membership functions  and ], respectively.Then their intersection  ∩  has a membership function Theorem 21 (see Liu [24]).Let  be an uncertain set with membership function .Then its complement   has a membership function Theorem 22 (see Liu [24]).Let  1 ,  2 , . . .,   be independent uncertain sets with inverse membership functions is an uncertain set with inverse membership function Definition 23 (see Liu [19]).Let  be a nonempty uncertain set.Then the expected value of  is defined by provided that at least one of the two integrals is finite.

Uncertainty Evaluation Method
When making a comprehensive evaluation, factors influencing the grade of the overall goal should be considered.The index system is often represented by a three-layer hierarchical structure with the overall goal, the second layer, and the bottom layer (namely, the factors influencing the overall goal's scaling).Experts always intend to show their own opinions and expectations of each evaluation index, so the evaluation results represent human uncertainty and belief degree.Therefore, the score value and weight of each evaluation index are represented by uncertain variables, and evaluation grades are represented by uncertain sets.Therefore, a new evaluation method based on uncertain variables and uncertain sets is proposed.The first level

Establishment of
The second level The third (or bottom) level The evaluation grades The overall goal A "Poor, " "Fair, " ..., "Very good" Figure 3: A simple hierarchical index structure.

Weights of
According to Definition 3, weight   is an uncertain variable and can be described by its uncertainty distribution Φ  .

Grade Vectors of
where   ( = 1, 2, . . ., ) is the membership degree to which index   belongs to grade   .The next step is to construct a method to realize the transformation from the grade vector   of the bottom index   to the grade vector () of the overall goal .

Transformation Method.
The transformation method can be obtained with the following three steps.
Step 1.To determine the importance of index   in the grading of the overall goal, the weight of index   with respect to the overall goal  is introduced beginning with the following formulas: where   () is the weight of index   with respect to .From the above formulas, we have Step 2. Because   () shows the degree of the effective information offered by   in the evaluation of , we calculate Step 3. By weight   (  ) of index   with respect to   , we have From the above formulas, we have Thus, we obtain the grade vector (  ) = ( 1 (  ),  2 (  ), . . .,   (  )) of index   in the second level, where   (  ) indicates the degree to which   belongs to evaluation grade   .
Remark 25.From the property of   (), it is obtained that the larger the   () is, the more important the role   plays in the evaluation of  is.If   () = 1, then the evaluation grade of  can be determined only by index   .If   () = 0, then index   does not play any role in determining the grade of .
Theorem 26.The weighted-average method is a special case of the above method.
Proof.For each row vector ( 1 ,  2 , . . .,  5 ), if there is one component   = 1 for some  and the rest four components are 0, then   () = 1,   () = 1, and   () = 1/.Thus, by formulas we have That is to say, the weighted-average method is a special case of the above method.
3.5.Identification.Sometimes, the evaluation scales are comparable (e.g., Poor, Fair,. .., Excellent scale; "Fair" is better than "Poor"), and a partial order "≻" can be defined according to the actual situation.If evaluation scale  1 is better than  2 , we denote  1 ≻  2 ; otherwise we denote  1 ≺  2 .Of course, whether  1 is better than  2 depends on the actual situation.
and if and then  belongs to   0 with at least the confidence level .

An Application in the Evaluation of Software Quality
In this section, an application of the proposed method in evaluation of software quality is given.

Evaluation System and Some Data.
A problem of evaluating software quality was discussed in Li [27].Next, we apply the proposed method to this evaluation problem, and the evaluation system is shown as in Table 2. Based on the evaluation criteria, the experts provided the scores of the bottom indices in Column 5 of Table 2.The evaluation grades are "Poor, " "Fair, " "Good, " "Very good, " and "Excellent." They are represented by uncertain sets  1 ,  2 , . . .,  5 with membership functions  1 ,  2 , . . .,  5 , respectively, where membership functions are given by experts according to their personal knowledge and actual situation of the evaluation.According to the scores given by experts (Column 5 in Table 2) and the above membership functions, the grade vectors (Column 4 in Table 2) can be obtained (see Section 4.3).
Similarly, the other grade vectors of the bottom indices can be obtained by the scores of the bottom indices (Column 5 in Table 2).

Grade
Vector of Software Quality .In this subsection, the transformation algorithm established above is used to realize the transformation from the grade vectors of the bottom indices to the grade vector of software quality.
(40) Therefore, the quality of this software is "Very good." 4.6.Comparative Analysis.In Li [27], the fuzzy max-min method is used in the software quality evaluation, and we calculate the results via the fuzzy weighted-average method with the same data in Table 2.The results are shown in Table 5.
Table 5 shows that the proposed method and fuzzy maxmin method produced the same result (i.e., the evaluation result is Very good) and the evaluation result from fuzzy weighted method is Good.And also there is some difference in the membership degree of each grade.Furthermore, the literatures [28,29] discussed the algorithms for maximizing the soft margin, which show that the larger the difference between two adjacent grades is, the stronger the classification ability of the method is.Therefore, we analyzed the differences between two adjacent grades in Table 6.From Table 6, we can see that there are three differences (between  4 and  5 ,  3 and  4 , and  2 and  3 ) and the proposed method is larger than the other two methods.Thus, the proposed method has a stronger classification ability in the evaluation of software quality.

Conclusions
In this paper, weights for bottom indices with respect to the overall goal in an evaluation system are introduced and a new uncertainty evaluation method is proposed based on uncertain sets and uncertain variables.This method generalizes the weighted average method, and it is applied in the evaluation of software quality.Comparative analysis with other two methods shows that the proposed method has a stronger classification ability in the evaluation of software quality.More importantly, the proposed method in this paper can also be used in other evaluation systems with a hierarchical structure.Therefore, more applications of the proposed method can be underdeveloped further.

Table 1 :
Four features of students  1 and  2 .

Table 2 :
Evaluation system of software quality.

Table 3 :
Evaluation results given by experts.

Table 5 :
Comparison of the results with the other two methods.