Quantitative Evaluation and Case Study of Risk Degree for Underground Goafs with Multiple Indexes considering Uncertain Factors in Mines

The accidents caused by underground goafs are frequent and destructive due to irregular geometric shapes and complex spatial distributions, which caused severe damage to the environment and public health. Based on the theories of uncertaintymeasurement evaluation (WME) and analytic hierarchy process (AHP), the comprehensive risk evaluation of underground goafs was carried out using multiple indexes. Considering the hydrogeological conditions, mining status, and engineering parameters of underground goafs, the evaluation index system was established to evaluate the risk degrees considering quantified uncertain factors. The single index measurement values were solved by the semiridge measurement function.The weights for evaluation vectors were calculated through the entropy theory and AHP. Finally, the risk level was evaluated according to the credible degree recognition criterion (CDRC) and the maximum membership principle. The risk levels of 37 underground goafs in Dabaoshan mine were evaluated using 4 coupled methods. The order for underground goafs risk degrees was ranked and classified on account of the uncertainty important degree. According to the ranked order, the reasonability of 4 coupledmethods was evaluated quantitatively. Results show that the UME-CDRC can be applied in the practical engineering, which provides an efficient guidance to both reduce the accident risk and improve the mining environment.


Introduction
Underground mining projects are the basic energy source and resource guarantee of the national economic development.The mining industry develops rapidly in the countries like South Africa, Australia, Canada, Poland, and China.With the large land area and relatively small population, it is viable to carry out the open pit mining in Australia and Canada, where little underground goafs are distributed [1][2][3][4].Compared to these countries, a large number of underground goafs formed in the process of mining in China, especially in the underground mines using open stope mining methods [5][6][7].Since the 1980s, the illegal mining was rising up in China, which caused numerous unknown underground goafs around the mines.There is no doubt that they present a very serious threat to safety and viability of efficient and sustainable mining, due to the characteristics of irregular geometric shapes, complex spatial distributions, and being strongly concealed [8][9][10].
In recent years, many collapse accidents of underground goafs have occurred, which have caused huge economic losses, casualties, and serious ecological damage.Particularly, the damage to our vegetation, water, and land from numerous underground goafs induced by large-scale mining has posed severe threats to the environment and public health [11][12][13][14][15][16][17].There are more than 9000 state-owned mines in China.The annual amount of mineral production has reached 9 billion tons and the volume of underground goafs has reached billions of cubic meters.In Shanxi Province, the area of underground goafs has been as large as 1/8 of the whole provincial area.Numerous mines in China are mining with underground goafs, including Dachang mine, Changba leadzinc mine, Shizishan copper mine, Luanchuan molybdenum mine, Lanping lead-zinc mine, and Dabaoshan mine.The Figure 1: The damage of roads caused by the collapse of underground goafs in Shengda iron mine, where the vibration intensity was equal to an earthquake with the magnitude 3.1.
Figure 2: The huge and numerous collapse pits distributed in the Hulun Buir, where the vegetation, environment, and ecological system were damaged seriously.
distribution range of underground goafs is extremely large and the locations are strongly concealed, which threatens the safety of mine production and surrounding environment.
The statistical results show that more than 180 places with large area of ground subsidence have been found in 21 provinces of China, which have caused more than 1592 collapse pits and more than 200 deaths.Based on the analysis of these accidents, two significant aspects for the harm of underground goafs are summarized [18].One aspect is the large area falling of underground goafs roofs, which results in the subsidence and cracking of surface, the destruction of mine environment, and heavy casualties.For example, a ground subsidence accident occurred at Shengda mine [19].The nearby roads and buildings were damaged with different levels by an induced vibration, where the vibration intensity was equal to an earthquake with the magnitude 3.1 (Figure 1).In addition, a large number of collapse pits were found in Hulun Buir (Figure 2).The vegetation was damaged seriously due to the large-scale coal mining, which led to the environmental disruption and imbalance of the ecological system [20].Another aspect is the groundwater inrush accident caused by numerous underground goafs and blasting vibration.Since, in the mining process, the rock fractures in the surrounding rock of underground goafs will develop due to the impact of blasting vibration and even connect to the surface or the chamber which contains water fully, thus the tunnels and working face will be inundated and causing huge losses.For example, a groundwater inrush accident occurred at Nandan, Guangxi Province, on 7 July 2001.In this disastrous accident, two mines were inundated at the same time and 81 people were killed.The volume of groundwater inrush was more than 34×10 4 m 3 and the direct economic losses reached 80 million yuan [21].
Furthermore, the environmental engineering geological problems induced by underground goafs also have a serious impact on the mine production, transportation, pipeline, land, and water movement.It is typical to note that there are a total of 1982 locations with different degrees of ground subsidence in the Fankou lead-zinc mine, Guangdong Province.The area of ground subsidence has reached 675 km 2 and the area of damaged farmland is about 66.7 km 2 .The area of the removed surrounding buildings is 7 km 2 .Besides, a national highway between Laizhou and Zhaoyuan suffered serious ground subsidence, which resulted in the destructions of traffic and surrounding buildings.At the same time, the adjacent underground goafs are more likely to penetrate each other and cause large-scale collapse of domino type [22].Thus, the evaluation of underground goafs risk degrees is a significant and urgent problem.
However, the evaluation of underground goaf risk degree is carried out through the single index in the existing methods, where the area of underground goafs is considered commonly [32].The application of comprehensive evaluation method for underground goaf risk degree using multiple indexes is scarce.As for the existing risk evaluation methods considering multiple evaluation indexes, most of them perform risk evaluation through the safety checklist, which is of qualitative evaluation.Actually, the weights of evaluation indexes will be affected seriously due to the subjective judgment, which leads to the inaccuracy and deviation of evaluation results.Recently, some comprehensive evaluation methods have been developed including grey fixed weight clustering method, grey relational analysis method, and neural network method [21,[33][34][35].In fact, the difficulty for evaluating underground goaf risk degree lies in the uncertainties and concealments of the influencing factors, which should be considered and analyzed together.Thus, a comprehensive method for evaluating the underground goaf risk degree, as well as ranking the order, is necessary and important.According to the ranked order of underground goafs risk degrees, it is feasible to provide theoretical support for the management order of underground goafs with higher safety hazards.Then, the underground goafs with severe safety threats can be managed timely, to avoid casualties and environmental destruction.
In this paper, UME and AHP theories are applied to evaluate underground goafs risk degrees comprehensively [36][37][38].The recognition criteria CDRC and TMMP are used to evaluate the risk levels of underground goafs.The multiple factors that influence the stability of underground goafs are considered and integrated comprehensively, which include the quantitative and qualitative factors, instead of a single factor.In addition, the quantitative and qualitative analysis are combined, and the qualitative factors are calculated with the quantitative form.According to the comparison and analysis of the 4 coupled methods formed through the combination of UME, AHP, CDRC, and TMMP, it is expected to develop a reasonable method for the evaluation of underground goaf risk degree, which is beneficial for the protection of environment and the sustained development of green mining.

The Index System for the Evaluation of Underground
Goaf Risk Degree.The underground goafs are of irregular geometric shapes and complex spatial distributions, where numerous factors have effects on the risk degrees.Based on the analysis of factors that influence the stability of underground goafs, 9 factors are selected as the qualitative single evaluation indexes including rock mass structure, geological structure, groundwater, impact of groundwater, mining impact, condition of adjacent underground goaf, pillar size and layout, the size of underground goaf (the ratio of the span to the height), and engineering layout, which are represented with  1 ,  2 ,  4 ,  5 ,  6 ,  7 ,  11 ,  13 , and  14 , respectively.In addition, 5 factors are selected as the quantitative single evaluation indexes including rock quality designation (RQD), span, area, height, and buried depth, which are represented with  3 ,  8 ,  9 ,  10 , and  12 , respectively.Tables 1 and 2 show the classification criteria of the 9 qualitative indexes and the 5 quantitative indexes, (i) For the underground goafs that belong to the risk level  1 , they should be monitored on schedule.The management process should be accomplished within twelve months.(ii) For the underground goafs that belong to the risk level  2 , they should be closed and separated.In addition, the distinct sign should be set up and the filling operation should be prepared in the meanwhile.The whole process should be accomplished within six months.(iii) For the underground goafs that belong to the risk level  3 , the distinct sign should be set up.The filling operation should be carried out within three months.(iv) For the underground goafs that belong to the risk level  4 , they should be closed and filled immediately.Moreover, the caving method should be applied as long as the safety requirements are satisfied.The whole process should be accomplished within one month.

Determination of the Single Index Measurement Value.
The underground goaf to be evaluated is represented as the symbol .The variable   represents a single evaluation index.As the number of selected evaluation indexes is 14, the evaluation index space is represented as  = { 1 ,  2 , . . .,  14 }.For the underground goaf  to be evaluated,   is the measured value corresponding to the single evaluation index   .Therefore, the parameter  can be represented as a vector  = { 1 ,  2 , . . .,  14 }.It is known that there are 4 risk levels for the underground goaf.The th risk level is represented as   ( = 1, 2, 3, 4) and it is safer than the ( + 1)th risk level, which can be rewritten as   <  +1 .The degree that the measured value   belongs to the risk level   is represented as   =  (  ∈   ).The parameter  is exactly the single index measurement value, which is determined through the single index measurement function.
Based on the analysis and comparison of different single index measurement functions, the semiridge distribution is selected to calculate the single index measurement values, which is shown as follows: where  and  are the lower and upper limit of the value range.Equation ( 1) is the rising semiridge distribution for the quantitative positive indexes, where the risk levels of underground goafs increase with the increase of single evaluation index.Equation ( 2) is the descending semiridge distribution for the quantitative negative indexes, where the risk levels increase with the decrease of single evaluation index.The semiridge functions of the 9 qualitative evaluation indexes and 5 quantitative evaluation indexes are shown in Figure 3. Thus, it is feasible to solve the single index measurement values of 14 evaluation indexes.The evaluation matrix (  ) 14×4 of single index measurement value is shown as follows: ] . (3)

Determination of Evaluation Index Weight.
Based on the summary of UME theory proposed by Wang [37], we improve the theory to fit the evaluation of underground goafs risk degrees.It is assumed that   is a parameter which represents the relatively important degree between the evaluation index   and other indexes.The parameter   can be taken as the weight of the evaluation index   , which should satisfy the requirements that 0 ≤   ≤ 1 and ∑ 14 =1   = 1.(c), (d), and (e) show the functions of RQD, span, area, height, and buried depth, respectively.The graph (f) shows the function of rock mass structure, geological structure, groundwater, impact of groundwater, mining impact, condition of adjacent underground goaf, pillar size and layout, and engineering layout.
The weight   can be determined by the entropy theory through the following equations: The evaluation matrix (  ) 14×4 of single index measurement value is known, and the weight   can be determined through (4).

The Comprehensive Evaluation Vector of Multiple
Index Measurements.The degree of the underground goaf  belongs to the risk level   , which is represented as   =  ( ∈   ).It can be obtained through It is obvious that the requirements 0 ≤   ≤ 1 and ∑ 4 =1   = 1 are satisfied here.Therefore, { 1 ,  2 ,  3 ,  4 } is the comprehensive evaluation vector of multiple indexes measurements for the underground goaf .

Risk level of underground goaf
Environmental factors

Hydrological factors
Geometrical parameters e criteria hierarchy e goal hierarchy e alternatives hierarchy Figure 4: The hierarchical model for AHP, which consists of the goal hierarchy, the criteria hierarchy, and the alternatives hierarchy.

Establishment of a Hierarchical Model.
According to the ideas and methods of AHP presented by Orencio and Fujii [36], the hierarchical model is established as shown in Figure 4, which consists of the goal hierarchy , the criteria hierarchy , and the alternatives hierarchy .The problem to be solved is taken as the goal hierarchy , which is the risk level   of the underground goaf  in this paper.The criteria hierarchy  is determined based on the analysis of the practical problem and the relationships between single evaluation indexes, which consists of geological factors, hydrological factors, environmental factors, and geometrical parameters.Specifically, the geological factors contain  1 ,  2 , and  3 .The hydrological factors contain  4 and  5 .The environmental factors contain  6 and  7 .The geometrical parameters contain  8 ,  9 ,  10 ,  11 ,  12 ,  13 , and  14 .All the single evaluation indexes make up the alternatives hierarchy, which is the lowest hierarchy.

Determination of the Judgment Matrix.
According to the importance degree for the 4 criteria in the middle hierarchy , the relative importance degree can be determined by 1-9 comparison scale method, which is clarified as follows.The number 9 indicates the greatest importance degree, while the number 1 indicates the smallest importance degree.It is assumed that   (1 ≤ ,  ≤ 4) is the judgment value of the criterion   with respect to the criterion   , which indicates the importance degree.On the contrary,   is the judgment value of the criterion   with respect to the criterion   , which is equal to the reciprocal of   .The detailed steps for determining   and   are shown in Table 3.Through (6), the judgment values can be solved by comparing   and   .Therefore, the judgment matrix  for the goal hierarchy can be obtained, which is shown as (7).Then, the greatest characteristic root and corresponding normalized feature vector of the judgment matrix  can be solved.Similarly, the judgment matrixes, greatest characteristic roots, and corresponding normalized feature vectors of the criteria hierarchies  1 ,  2 ,  3 , and  4 can be solved.

The Consistency Check of the Judgment Matrix.
The consistency check should be performed for the judgment matrixes to determine whether they are qualified.According to (8), the consistency index (CI) is solved.The average random consistency index (RI) can be found from Table 4.
Then, we can calculate the consistency ratio (CR) through (9), which is equal to the ratio of CI to RI.The consistency of the judgment matrix is qualified when there is CR < 0.1.
Otherwise, the judgment matrix should be determined again with different criteria.
where  is the number of rows for the judgment matrix to be checked.

Determination of the Synthetic Weight.
As the judgment matrixes, the greatest characteristic roots, and the corresponding feature vectors of ,  1 ,  2 ,  3 , and  4 are known, the consistency check can be performed for these judgment matrixes.The results of the judgment matrixes, the greatest characteristic roots, and the corresponding feature vectors are shown in (10).The synthetic weights of all the evaluation indexes are represented as   , which are shown in Table 5.

The Recognition Criteria for UME and AHP.
In order to make sure that the evaluation results for underground goafs are reasonable and reliable, CDRC and TMMP are applied to compare the evaluation results and verify the reasonability.

The Credible Degree Recognition Criterion (CDRC).
The credible degree is represented as  ( ≥ 0.5).It is credible that the underground goaf  belongs to the evaluation level   when the following requirements are satisfied:

The Maximum Membership Principle (TMMP).
The membership for the evaluation level   of the underground goaf  is exactly the corresponding weight.In UME method, it is equal to   , whereas it is equal to   in the AHP method.It is credible that the underground goaf  belongs to the evaluation level   when the corresponding weight is the maximum among all the weights.

The Method for Ranking the Order of Underground Goafs.
There is not only a need to evaluate the risk level   of the underground goaf , but also a requirement to rank the order for many underground goafs according to the risk degrees.If there is  1 <  2 <  3 <  4 and   is the score value of   , then we can obtain the following equations: where   is the uncertainty important degree of the underground goaf .Therefore, the order for risk degrees of many underground goafs can be ranked by the values of   .

The Evaluation Process for Risk Levels of Underground
Goafs.Firstly, the single index measurement value can be solved based on the semiridge distribution.Then, we can obtain the evaluation matrix (  ) 14×4 of single index measurement value.Secondly, the weights of single evaluation indexes in UME can be solved through the entropy theory.
In the AHP theory, all the weights can be solved through the AHP.Then, it is feasible to obtain the comprehensive evaluation vectors of UME and AHP.Finally, the recognition criteria including CDRC and TMMP are applied to evaluate the risk level of the underground goaf.Therefore, there are 4 coupled evaluation methods for the risk level through the combination of UME, AHP, CDRC, and TMMP.The comprehensive evaluation process for the risk level   of the underground goaf , which mainly includes UME-CDRC, UME-TMMP, AHP-CDRC, and AHP-TMMP, is shown in Figure 5.

The Dabaoshan Mine.
The Dabaoshan mine is located in Guangdong Province, China, where there is a superimposition place of one fault belt and another tectonic belt.Due to the open stope mining and illegal mining, numerous underground goafs formed and caused the continuous increase of geostress.Figures 6(a) and 6(b) show the distinct geostatic activities such as pillar cracking and roof falling, which threaten the safety of ground, slope, and underground production system.In the mining process, the sizes of some underground goafs were large, while the sizes of pillars were too small to bear the pressure.Then, the pillar cracking and the large area falling caused a serious landslide accident, which caused threat and destruction to workers and environment.Figure 6(c) shows the landslide accident caused by the collapse of underground goafs.The statistical results of Dabaoshan mine show that the number of underground goafs is 254, and most geometric shapes are irregular.The volume of the smaller underground goaf is about 400 m 3 , whereas the larger volume is greater than 10 × 10 4 m 3 and the height is greater than 100 m.In view of the current status, the evaluation of underground goafs risk degrees is important and urgent.
Combined with the hydrogeologic conditions, environmental factors, mining status, and structural parameters of underground goafs, a total of 37 underground goafs are selected to evaluate the risk levels.The data for 14 single evaluation indexes of 37 underground goafs are listed in Table 6 [39].According to the estimated and measured data provided in Table 6, as well as the single index measurement values solved in Figure 3, the evaluation matrixes (  ) 14×4 of single index measurement value for the 37 underground goafs can be obtained.The underground goaf  1 is taken as an example to calculate the evaluation matrix (  ) 14×4 of single index measurement value, which is shown as follows: The weights w of evaluation indexes 1.0000 0 0 0 1.0000 0 0 0 0 0 1.0000 0 0 0 1.0000 0 0.3087 0.6913 0 0 0 0 1.0000 0 0.9797 0.0203 0 0 0 0 1.0000 0 0.1654 0.8346 0 0 1.0000 0 0 0 1.0000
Based on the recognition criterion CDRC, it can be obtained that  = 0.0760 + 0.3946 + 0.1494 = 0.6200 >  = 0.5 when the credible degree  is set as 0.5.Thus, the risk level of the underground goaf  1 is level  3 .The risk level is evaluated as level  2 while applying the recognition criterion TMMP.
According to the AHP theory, the weights of the criteria hierarchy and the alternatives hierarchy, as well as the The evaluation vectors using UME and AHP of all the 37 underground goafs, as well as the risk levels evaluated by CDRC and TMMP, are listed in Table 7.The comparison results for risk levels evaluated by CDRC and TMMP of 37 underground goafs are shown in Figure 7.

The Results for Ranking the Order of Underground Goafs.
As   is the score value of   and the risk level   is safer than  +1 , it can be assumed that  1 = 1,  2 = 2,  3 = 3, and  4 = 4. Then the uncertainty important degree  of the underground goaf  can be solved according to (13).The uncertainty important degree of UME is represented as  UME , whereas the uncertainty important degree of AHP is represented as  AHP .The results of  UME and  AHP are shown in Figure 8.Based on the calculation results, we can rank the order of risk degree for 37 underground goafs from lower to higher, which is   The uncertainty important degree 2 4 6 8 10 12 14 16  AHP have the similar ability and effect for ranking the order of risk degree.Furthermore, the ranking results can provide theoretical and reasonable guidance for the management of underground goafs.Actually, it is difficult to manage numerous underground goafs at the same time due to the great need of time, manpower, and environmental factors.Therefore, the underground goaf of higher risk degree can be managed firstly according to the order of risk degree.As a result, it is feasible to manage underground goafs reasonably without influencing the normal production.

Discussion
For the risk levels of underground goafs to be evaluated, there are 4 coupled methods, which are UME-CDRC, UME-TMMP, AHP-CDRC, and AHP-TMMP, respectively.Table 8 shows the underground goafs of four risk levels with 4 coupled methods.
It is obvious that the order of risk degree is higher and the risk level of the underground goaf is higher.As the ranking results are the same for both UME and AHP, we can arrange all the underground goafs along a straight horizontal line from the lower risk degree to the higher risk degree, which is shown in Section 3.3.Then, we can divide this straight line into four sets  1 ,  2 ,  3 , and  4 by setting three dividing lines artificially.In addition, the setting of three dividing lines is different based on the method to be analyzed.Also, the setting of dividing lines should be the optimal for every method.Therefore, there will be four sets of all the underground goafs with every method.Then, the underground goafs of four sets can be compared to the underground goafs of four risk levels quantitatively.The results shown in Table 8 are taken as standard data.Compared to the underground goafs of four sets  1 ,  2 ,  3 , and  4 , the detailed evaluation criteria are stated as follows: (i) If the risk level of an underground goaf is equal to the level of the divided four sets, then the score value for the underground goaf is 0.
(ii) If the difference between the risk level of an underground goaf and the level of the divided four sets is  equal to 1, then the score value for the underground goaf is −1.(iii) If the difference between the risk level of an underground goaf and the level of the divided four sets is equal to 2, then the score value for the underground goaf is −2.(iv) If the difference between the risk level of an underground goaf in and the level of the divided four sets is equal to 3, then the score value for the underground goaf is −3.
If the final score value is higher, it means that the corresponding method is more reasonable and suitable for the evaluation of risk level.UME-CDRC is taken as an example: the first dividing line is set between  13 and  18 , the second dividing line is set between  17 and  34 , and the third dividing line is set between  36 and  37 .Therefore, there are  8, it is obvious that only the risk level of the underground goaf  25 is different from the standard data, where the difference is equal to 1.Then, the final score value of UME-CDRC is determined as −1.
Similarly, the final score value of UME-TMMP, AHP-CDRC, and AHP-TMMP can be solved as −8, −5, and −7, respectively.The four sets and the corresponding score values of every method are listed in Table 9.
Obviously, the score value of UME-CDRC is the maximum among the 4 coupled methods.As a result, UME-CDRC is the most reasonable method for the evaluation of underground goafs risk degrees.Through the comparisons Geofluids of UME-CDRC and UME-TMMP, AHP-CDRC, and AHP-TMMP, the score value of UME-CDRC is much smaller than UME-TMMP and the score value of AHP-CDRC is smaller than AHP-TMMP.Thus, it can be concluded that CDRC is more reasonable than TMMP.Sometimes, the difference between the greatest weight and the second greatest weight in the evaluation vector is little.In this case, the risk level evaluated through TMMP will not be reasonable and reliable.On the contrary, CDRC is suitable for evaluation of underground goafs risk levels whatever the difference between the greatest weight and the second greatest weight.As for AHP, it is more subjective than UME because the process of establishing the hierarchical model and the determination of weights for evaluation indexes are more subjective.The evaluation vector of UME is determined through the measured values, which is more objective.Therefore, it is concluded that UME-CDRC is reasonable for evaluation of underground goafs risk degrees in the practical engineering application.

Conclusions
As the special characteristics of underground goafs are with irregular geometric shapes, concealed locations, and huge amount, they have caused numerous safety accidents and destructive disasters, which influence the normal production in mines and the surrounding environment.Therefore, the evaluation of underground goafs risk degrees is an urgent and significant problem.In this paper, the theories of UME and AHP were applied to comprehensively evaluate the risk levels of underground goafs.According to the hydrogeological conditions, mining status, and engineering parameters, 14 factors were selected to establish the evaluation index system.Based on the estimated and measured data, the semiridge distribution is selected as the single index measurement functions.Then, the single index measurement values were solved quantitatively according to the single index measurement functions.Furthermore, the weights of evaluation vector were determined by applying the entropy theory and AHP.Once the evaluation vectors of UME and AHP were obtained, the risk levels of underground goafs can be evaluated through the recognition criteria CDRC and TMMP.Also, the order of risk degree can be ranked per the values of uncertainty important degree.Combining the calculation methods UME and AHP with the recognition criteria CDRC and TMMP, there are 4 coupled methods UME-CDRC, UME-TMMP, AHP-CDRC, and AHP-TMMP.The data of underground goafs in Dabaoshan mine were used to evaluate the risk levels using the 4 coupled methods, as well as ranking the order of risk degree.On account of the ranked order, it is proved that UME-CDRC is the most reasonable method through both quantitative and qualitative analysis.Results show that UME-CDRC not only can evaluate the underground goafs risk degrees reasonably, but also can rank the order of risk degree, which insures that the most dangerous underground goaf can be managed at the first time to reduce the probability of accidents.The developed UME-CDRC provides a reasonable and reliable methodology to evaluate the underground goafs risk degrees, which is not only a beneficial complement for the management methods of underground goafs, but also a big improvement for environmental protection as well as for the sustained development of green mining.

Figure 3 :
Figure 3: The single index measurement functions for evaluation indexes of underground goafs.The graphs (a), (b),(c), (d), and (e)show the functions of RQD, span, area, height, and buried depth, respectively.The graph (f) shows the function of rock mass structure, geological structure, groundwater, impact of groundwater, mining impact, condition of adjacent underground goaf, pillar size and layout, and engineering layout.
The maximum membership principle (TMMP) Credible degree recognition criterion (CDRC) UME-CDRC AHP-TMMP AHP-CDRC UME-TMMP Risk level C k for the underground goaf R The comprehensive evaluation vectors  of UME and  of AHP The evaluation matrix  ik 14×4 of the single index measurement value and the corresponding feature vectorsThe judgment matrixes G, P 1 , P 2 , P 3 , P 4

Figure 7 :
Figure 7: The comparison results for risk levels of 37 underground goafs.The graph (a) shows the risk levels of underground goafs using CDRC, and the graph (b) shows the risk levels using TMMP, where the blue part represents the risk levels of UME and the orange part represents the risk levels of AHP.

Table 1 :
Classification criteria for evaluation of risk levels with qualitative indexes.
1 is the ratio of span to height.

Table 3 :
The judgment values for determining the importance degree between the criteria   and   .

Table 4 :
The values for average random consistency index (RI). 12  13  14  21  22  23  24  31  32  33  34  41  42  43  44 ,  max = 7.0003, CR < 0.10.Determination of the Evaluation Vector.The degree that the underground goaf  belongs to the risk level   is represented as .Since the evaluation matrix (  ) 14×4 of single index measurement value and the weights of hierarchical model are known, the comprehensive evaluation vector { 1 ,  2 ,  3 ,  4 } can be obtained by multiplying the weights and evaluation matrix from the alternatives hierarchy to the goal hierarchy.

Table 5 :
The results of the weights in the hierarchical model.Weights of the criteria hierarchy () Weights of the alternatives hierarchy () The synthetic weights ()

Table 6 :
Estimated and measured data for single evaluation indexes of 37 underground goafs.

Table 7 :
The comparison results for UME and AHP of 37 underground goafs, as well as the risk levels evaluated by CDRC and TMMP.The risk level is evaluated as level  3 by CDRC, while the risk level is evaluated as level  4 by TMMP.