A Novel Hybrid Evaluation Model for the Performance of ERP Project Based on ANP and Improved Matter-Element Extension Model

Considerable resources are needed when implementing the ERP project, so it is necessary to evaluate its performance. Firstly, the evaluation index system of implementation performance of the ERP project was built, and an Analytic Network Process (ANP) which can fully take the relationship between evaluation indexes into account was employed to determine the index weight. Secondly, an improved matter-element extension model, which can overcome the limitations and inadequacies of traditional matter-element extension model when performing the comprehensive evaluation, was proposed to evaluate the implementation performance of the ERP project. Finally, taking an enterprise’s ERP project as an example, a comprehensive evaluation was done, and the empirical analysis result shows that this proposed hybrid evaluation model is feasible and practical.


Introduction
The Enterprise Resource Planning (ERP) system which is built on the information technology and systematic management thoughts can provide a decision-making management platform for enterprise's management team and staff.The ERP system plays a significant role in improving the business processes and competitiveness of an enterprise.The implementation of ERP changes the organizational and business mode, which brings a huge impact on each enterprise's department.Since considerable enterprise resources are needed when implementing ERP project, it is quite necessary to build a reasonable and effective comprehensive evaluation method to evaluate the performance of ERP project.
Many evaluation models have been used to evaluate project performance, such as analytic hierarchy process (AHP) [1], fuzzy analytic hierarchy process (FAHP) [2], data envelopment analytic hierarchy process (DEAHP) [3], balanced scorecard (BS) [4], and neural network analysis method [5].In addition, many studies have also been conducted on evaluating ERP project performance.Chen and Lin [6] proposed a fuzzy linguistic performance indicator based on network flow model to assess the performance of ERP systems.Zhan et al. [7] presented an evaluation model based on the Triangle Whiten Function.Xu [8] used an AHP method to evaluate the performance of ERP, considering the feedback and dependence factors.Razmi et al. [9] proposed a fuzzy analytic hierarchy process model (FAHP) which combines fuzzy theory with analytic hierarchy process to evaluate the ERP project.Chang et al. [10] constructed a conceptual model to measure the performance and competitive advantages of ERP from a supply chain management perspective.Hanet al. [11] used ABCD monitoring table and SPA project evaluation method to assess the ERP project.The AHP method does not consider the relationship between different indexes of control level in the index system, which weakens the objectivity of the evaluation result.BP neural network evaluation method does not determine the index weight but it requires a large amount of training samples.Although data envelopment analysis method is relatively objective, it is not suitable for qualitative analysis.ABCD monitoring table and SPA project evaluation method can cover a wide range of indexes, but the quantification of indexes is very difficult.Therefore, a more practical and objective evaluation method needs to be proposed.
The matter-element extension analysis model transforms practical problem into a formal one using matter-element and extension theory and presents grade of things through calculating the correlation between the matter-element to be evaluated and each level.In addition, the matter-element extension analysis can be also used though there are few samples.Li and Zhang [12] assessed the performance of the employees through establishing a qualitative and quantitative performance evaluation method based on the matter-element extension theory, and results shows that this method is more objective.Zhou [13] established a performance evaluation model based on matter-element and correlation function.Qualitative and quantitative evaluations on the performance of conglomerate merger were done.However, if index value exceeds the controlled field, correlation function cannot be calculated.Therefore, the traditional matter-element extension model needs to be improved.
In order to evaluate the performance of enterprise's ERP project, a hybrid evaluation model combining ANP and improved matter-element extension model was proposed.Firstly, the evaluation index system was built; secondly, an ANP was used to determine the weight of each index which fully took the relationship among various indicators into account; and then, an improved matter-element extension model which can overcome the limitations and inadequacies of traditional matter-element extension model was proposed.Finally, taking an enterprise's ERP project as an example, the comprehensive evaluation was done, and empirical analysis results show that this hybrid evaluation model is feasible and practical.

Building the Performance Evaluation Index
System of ERP Project ERP project involves many aspects of enterprise management, such as production management, financial management, sales management, purchasing management, and inventory management [14].Therefore, the implementation of ERP project not only relies on IT departments, but also relies on the collaboration of other departments in enterprise.Managers must be familiar with management and technical business based on ERP.Tasks are completed by professional staff, so the overall quality of employees, production efficiency, and production capacity will increase correspondingly.
Based on the financial system, financial capital operation reaches a dynamic equilibrium.Meanwhile, turnover rate of the total funds and enterprise's return on equity improve accordingly.
The implementation of the procurement and inventory system provides many inventory analysis methods for the supply department.This system can not only ensure the procurement of purchased parts timely, but also improve inventory levels, which can reduce the backlog of inventory funds and accelerate the efficiency, timeliness, and accuracy of the inventory turnover of delivery.The implementation of procurement and inventory system will raise overall level of operational management, market share, and new customer acquisition rate.
A good operation of ERP system makes data integral, accurate, consistent, and timely.Data sharing becomes accurate and timely accordingly.ERP project can also support business decision, improve forecasting production plan, and ensure a stable and efficient operation in an enterprise.

Building the Evaluation Index System.
In order to evaluate the implementation performance of enterprise's ERP projects accurately, it is necessary to establish a reasonable evaluation index system and grading standard.Based on former analysis about the effect of implementing ERP system, we use Delphi method to build a performance index system of ERP project [5,14].The finance management, operations management, and customer management are criterion layer indexes.In addition, 11 extended indexes are concluded in the subcriterion layer.The index system is shown in Figure 1.

Basic Theory of Extension Analysis.
Matter-element extension model [13,15,16] is based on matter-element theory and extension set theory.We can determine the level of one thing through establishing classical field, controlled field, evaluation level, and correlation function.However, there are some limitations and deficiencies.
(1) When any matter-element index value beyond its controlled field, the correlation function values are unavailable, so this model cannot perform evaluation.
(2) The level of one thing is obtained by calculating correlation function in this model.From the perspective of algorithm, correlation degree can be regarded as an extension of membership degree in fuzzy mathematics, so the correlation degree principle is equivalent to the maximum membership principle [15].In some case, however, the maximum membership principle cannot reflect the ambiguity of object's boundary.It will lose information and lead to the deviation of results.
Aiming at the limitation of (1) point the classical domain and the matter-element to be evaluated should be normalized.Aiming at the limitation of (2) point the maximum membership degree criterion should be replaced by the correlation degree criterion.method [13,15] is as follows: first of all, the object is divided into levels and the range of each level is determined by database or experts; secondly, determine the weight of each index; Finally, calculate the closeness degree and determine the level of matter-element.

The Establishment of the Improved Matter-Element Extension Model. The basic idea of matter-element evaluation
Matter element is the logic unit of matter-element extension model; it uses an ordered triple  = (, , ) to describe things., ,  represent the name, the characters, and the value of one thing, respectively.The basic steps of improved matter-element extension model are as follows.
(1) Determine the classical domain, controlled field, and matter element of the object to be evaluated.
Suppose the classical domain matter element is as follows: where P  represents the th grade;  1 ,  2 , . . .,   are  different characteristics of P  ; and V 1 , V 2 , . . ., V  are the value ranges of P  about  1 ,  2 , . . .,   , respectively, namely, the classical field.
Suppose the controlled field matter element is as follows: where  represents all grades of the object to be evaluated, and V 1 , V 2 , . . ., V  are the value ranges of  about  1 ,  2 , . . .,   , namely, the controlled field of .Suppose the matter element to be evaluated is as follows: where P 0 represents all grades of the object to be evaluated, and V 1 , V 2 , . . ., V  are actual data of P 0 about  1 ,  2 , . . .,   .
When an actual value of index exceeds the controlled field, the correlation function cannot be calculated, namely, the denominator is zero.In this case, matter-element and extension model cannot be used to evaluate the performance of ERP project.Therefore the classical domain and matterelement evaluation should be normalized.
Normalize the classical domain R  as follows: Normalize the matter-element evaluation R 0 as follows: (3) Weight determination.
The Weight of the evaluation index directly affects the quality and feasibility of a comprehensive evaluation.So the determination of index weight is very important to the result of enterprise performance comprehensive evaluation.Since the evaluation index system is complex and related, an Analytic Network Process is used to determine the weight of each index, which can fully take all characters of indexes into account.
(4) Establish and calculate the closeness function.
Zhang [17] used closeness degree criteria instead of maximum degree of membership criteria, and made a theoretical analysis.He put forward an asymmetric closeness formula ( = 1) as follows: where  represents the value of closeness function;  represents the distance;   is the weight.The value of closeness function about each index of the matter element to be evaluated with each level is calculated as follows: where D  (V  (5) Rating.
Suppose    ( 0 ) = max{  ( 0 )}, the matter element to be evaluated  0 belongs to the   th level. Suppose where  * represents the level variable eigenvalue of  0 .The attributive degree of the evaluated matter-element tending to adjacent levels can be judged from  * .

Analytic Network Process Method. Analytic Network
Process [18][19][20] was developed from analytic hierarchy process.This method fully considers the interdependence between elements, mutual influence between elements in the same level, and dominance relation from the lower level.All elements form a network structure of ANP.An ANP consists of two parts.The first part is the control layer, including goal and criterion.In this layer, each criterion is independent and controlled only by target element.The second part is the network layer; it is controlled by control layer, and the elements in the network layer influence each other (Figure 2).

Basic Operation
Taking   as standard, we compare the influence degree of each element to   and we can get weighted matrix as follows: The matrix  is multiplied by the matrix ; then we get the weighted matrix:  =     ,  = 1, 2, . . .,  ,  = 1, 2, . . ., .If the limit of matrix  is convergent and only  ∞ is the limit relative ranking vector of each element to the  element under the   principle.

The Calculation Process of the ANP-Improved Matter-Element Extension
Model.In summary, the steps of comprehensive evaluation based on the ANP and improved matterelement method are as follows.
Step 1. Determine the classical domain, controlled field, and matter element to be evaluated.
Step 2. Normalize the classical domain, controlled field, and matter element to be evaluated, when the measure data of index exceeds controlled field.
Step 3. Calculate the index weight based on the ANP.
Step 4. Calculate the value of closeness function of each grade.

Case Study
A manufacturing enterprise began to implement ERP project two years ago.In order to figure out the situation of this project, an evaluation of ERP project performance was proposed.The evaluation process is as follows.
4.2.Calculate the Index Weight.An ANP was used to determine the weight of performance evaluation index, which can fully take the relationship between various indicators into account.Because to the calculation is complicated, we employed "Super Decision" software to calculate the weight of each index.The steps are as follows.According to the index system diagram in Figure 1, the control level and network level are developed based on the dominance and feedback relationship.The dependencies among indexes in the control level are shown in Figure 3.The relationships among indexes in network level are depicted in Figure 4.An ANP model is developed on the network structure of elements.
Based on the clear relationship between elements, the judgment matrixes are concluded through comparing element sets and elements, respectively, by nine-scale method.For example, the pairwise comparisons of elements in clutter B1 are conducted and the judgment matrix is shown in Table 1, and the local weight can be concluded by "Super Decision" software.
(3) Check the consistency of judgment matrix.
If CR ratio exceeds 0.1, we should change the judgment matrix until all CR ratios are less than 0.1.
(4) Calculate the weighted supermatrix and the limit matrix.
Calculate the weighted supermatrix and the limit matrix of indicator elements by software.The results are shown in Figures 5 and 6.     (5) Calculate the weight of indexes according to the weighted supermatrix and limit matrix.The results are shown in Table 2.

Calculate the Value of Closeness Function.
Calculate the distance   (V   ) of the evaluated matter element related to new classical domain, just as shown in Table 3.
The value of the closeness degree of each grade is below: 4.4.Determine the Performance Rating.Since  1 ( 0 ) = max{N  ( 0 )} = 0.999482,  = 1, 2, 3, 4, it is shown that the performance level of ERP project in this enterprise belongs to "high." 4.5.Sensitivity Analysis.Sensitivity analysis is performed according to the performance evaluation index system of ERP project.When the index value or weight of index changes, its level variable characteristic value  * also changes correspondingly.When the index value changes by 5%, 10%, −10%, −20%, −30%, −40%, −50%, respectively, the level variable characteristic value  * changes as shown in Figure 2. When the weight of index changes by ±10%, ±20%, ±30%, ±40%, or ±50%, the level variable characteristic value  * changes correspondingly, just as the Figure 3.
As we can see from Figure 7, the project performance indexes  1 ,  2 ,  8 ,  11 have large effect on the evaluation results, which means that the sensitivity of  1 ,  2 ,  8 ,  11 is very strong.For example, when the  11 index changes, the range of the  * value varies from 1.2 to 1.8.Although ERP project performance level does not change, the level gradually tends to "good" level from "high" level.With the change of index  3 ,  4 ,  5 ,  6 ,  7 ,  9 ,  10 , the  * value changes a little from 1.5 to 1.65.It indicates that the variation of index value will not change the level which the project performance tends to or belongs to.
In Figure 8, we can know that the change of  1 and  11 indexes affects the  * value obviously, but other indexes have less effect on the  * value.In a word,  1 and  11 are sensitive indexes in the ERP project performance evaluation.

Conclusions
ERP system has been introduced into enterprises for many years.It is necessary to evaluate the performance of the ERP project in enterprise.However the factors which affect the performance of ERP project are complex and related.So a hybrid evaluation method of ERP project performance which considers these peculiarities was proposed.In order to analyze the performance of ERP project, an index system of comprehensive benefit evaluation including finance, customer, and operation management is established in this paper.An ANP was proposed to determine the weight of each index which fully took the relationship between various indicators into account.Due to the limitations and inadequacies of traditional matter-element extension model when performing the comprehensive evaluation, an improved matterelement extension model was proposed.And then, taking one enterprise's ERP project as an example, the comprehensive evaluation was done.The empirical analysis results show the performance of ERP project in our case belongs to "high" level and our proposed hybrid evaluation model is feasible and practical.Finally, a sensitivity analysis is performed to find sensitive index in the ERP project evaluation.The analysis results show that total asset turnover ratio and data transfer efficiency are sensitive indexes, so this enterprise should pay more attention to them.

Figure 2 :
Figure 2: The basic structural diagram of ANP.

Figure 6 :
Figure 6: The limit matrix of indicator elements.

Figure 7 :Figure 8 :
Figure 7: Variation of  * with the index values.
) represents the distance of matter element to be evaluated related to its corresponding normalized classical field;   () represents the weight of evaluation index;  represents the number of evaluation index.

Table 1 :
The judgment matrix of elements.

Table 2 :
The global weights of each criterion.