While TOPSIS is a widely used evaluation method, it lacks evaluation from the perspective of indicator coordination. Based on the analysis of TOPSIS evaluation methods, this paper proposes a new assessment method, coordinated TOPSIS, which takes into account the advantages of TOPSIS evaluation methods and the coordination level of evaluation indicators. Taking the robotics academic journals as an example, the TOPSIS evaluation and the coordinated TOPSIS evaluation results are compared. The research shows that the weights of the coordinated TOPSIS method can be manually adjusted according to the purpose of evaluation and have a good flexibility; the TOPSIS evaluation results and coordinated TOPSIS evaluation results have a high degree of correlation, but the difference from the perspective of the ranking is big; the coordinated TOPSIS is very suitable for evaluation field that needs to consider the coordinated development, and it can be replicated.
It is of great significance to evaluate academic journals of robotics. With the rapid development of artificial intelligence, the development of robotics disciplines is very fast. In recent years, robotic academic journals have become increasingly influential, and their influence factors are getting higher and higher, which makes it very important to promote the development of robotics disciplines. The evaluation of robotic academic journals not only helps the competition between different journals and improves the academic quality, but also facilitates the selection of appropriate journals by the author when posting. It provides great value to journals and scholars and they can jointly support the development of robotics disciplines.
TOPSIS is a widely used evaluation method. Since Huang proposed the TOPSIS evaluation method [
In the evaluation of science and technology, TOPSIS has also been widely used. Xu et al. use the TOPSIS method to evaluate the output of scientific research institutions [
Just as any evaluation method has its own advantages and disadvantages; TOPSIS also has some room for optimization. Jahanshahloo et al. study the distance calculation formula of the TOPSIS method when the attribute data is a fuzzy number [
Sun et al. evaluate the impact of green technology innovation on ecological-economic efficiency of strategic emerging industries with the entropy weighted TOPSIS method [
First, evaluation of academic journals can be done from a variety of perspectives, including academic quality, journal influence, timeliness, and editorial quality. An excellent journal should strive to achieve comprehensive and coordinated development. Each evaluation index should not be lacking. However, for the current TOPSIS evaluation method, when certain indicators are low, they can be complemented when the other indicators are high.
Second, the coordinated development of academic journals is also in line with the laws of journal development. For example, the timely delivery of the robot journals is efficient, which is a manifestation of their academic quality. Editing and publishing excellence is also an integral part of journal quality. Journals with better academic quality are bound to have higher influence.
Third, there are not many multiattribute methods which evaluate from the perspective of coordination. There are dozens of multiattribute evaluation methods. TOPSIS is only one of the evaluation methods. When TOPSIS is designed initially, the coordination of evaluation indicators is not considered. If improved, it can not only optimize the TOPSIS evaluation method and enrich the basic theory of evaluation but also have important practical significance for the evaluation of academic journals. It can also promote the all-round and balanced development of academic journals.
Based on the analysis of TOPSIS method principle, this paper proposes a coordinated TOPSIS evaluation method. Taking the JCR2016 Robotics Journal as an example, this paper studies the differences between the evaluation method and the original TOPSIS evaluation method and then summarizes the results.
The traditional TOPSIS calculation formula is
In formula (
Figure
Coordination issues for evaluation.
As shown in Figure
As shown in Figure
Calculation of coordination degree.
In order to find out the size of ∠NOA, you need to calculate the cosine of the angle first and then convert it by the inverse cosine function. Yu and Zhang proved that the length of OB is (
It can be utilized across the board. Assuming in general ∠NOA=
The value of
This method can also be applied to weighted TOPSIS. There is no concept of weighting when the TOPSIS method was developed. Scholars such as Shyur, Deng et al., Yue and Zeng, and Xiao (2018) introduced weights to TOPSIS. The indicator coordination line of the weighted TOPSIS is not 45. straight line [
Weighted TOPSIS reference.
In the weighted TOPSIS evaluation, the
Due to the weighted TOPSIS, the reference baseline is not 45.. Therefore, when calculating the degree of coordination, it is better to convert using 90, that is,
To revise TOPSIS from the point of view of coordinated development, two issues must be considered. First, the TOPSIS evaluation results must be considered to reflect the superiority and characteristics of TOPSIS, and the second is coordination. As for the synthesis of the two, there are two ideas, one is to take additive synthesis, and the other is multiplication synthesis. For the multiplication synthesis, for points on X-axis or Y-axis, which are completely uncoordinated, the coordination degree is 0, so the overall evaluation value is 0, which is not in line with the usual; therefore additive synthesis is adopted. The weight of the coordination degree can also be set according to the purpose of the evaluation. For this purpose, the coordination weight v is introduced, 0<v<1. If the assessment is to encourage the coordinated development of the evaluation indicators, v must be greater than or equal to 0.5.
In addition, we must pay attention to the dimensional issue of the TOPSIS evaluation value and coordination degree. In TOPSIS evaluation, since the positive and negative ideal solutions are often not in the same evaluation object, the theoretical maximum value is 1 and the minimum value is 0. However, this situation often does not exist in actual evaluation. In order to combine TOPSIS evaluation values and coordination degree, the TOPSIS evaluation results must be standardized. In the same way, the degree of coordination also needs to be standardized. This coordination formula is
In this equation,
In TOPSIS evaluation, if a negative ideal solution is used, the line between negative ideal solution to the ideal solution will not normally be 45°. It is not meaningful to use the non-45° line as a criterion to judge whether or not the evaluation indicators are coordinated. This is because the coordination reference standard is not unique. In this circumstance, a more scientific approach is to adopt an absolute negative ideal solution, that is, to use the origin as a negative ideal solution. In addition, the use of an absolute negative ideal solution can also fundamentally eliminate disorder problem in TOPSIS. The sequences of all evaluation results will be changed if the negative ideal solution changes.
It should be noted that, in the following empirical research, in order to conduct a comparative analysis of TOPSIS and coordinated TOPSIS, absolute negative ideal solutions are used in TOPSIS evaluation, and the value at the origin of the coordinate axis is the worst value.
This article takes the JCR2016 Robotics Journal as an example to compare and analyze TOPSIS and coordinated TOPSIS evaluation results. The robotics discipline is one of the few disciplines in JCR 2016, with a total of 26. Since the four journals “IEEE Transactions on Cognitive and Developmental Systems,” “Intelligent Service Robotics,” “IEEE Transactions on Autonomous Mental Development,” and “Frontiers in Neurorobotics” have missing data, they are not considered, so there are actually only 22 journals.
JCR2016 published a total of 11 indicators, namely, Total Cites, Journal Impact Factor, Impact Factor without Journal Self Cites, 5-Year Impact Factor, Immediacy Index, Cited Half-Life, Citing Half-life, Eigenfactor Score, and Article Influence Score. Average Journal Impact Factor Percentile, Normalized Eigenfactor, and Since Normalized Eigenfactor is derived linearly from characteristic factors, it is removed to avoid data duplication, Average Journal Impact Factor Percentile is calculated based on the impact factor, and it belongs to the ranking index, which is discontinuous numerical index. Therefore, it is also deleted. Nine indicators are used for evaluation, and the statistics of the data are shown in Table
Index description statistics.
Evaluation index | Mean | Median | Maximum | Minimum | Std. Dev. |
---|---|---|---|---|---|
Total Cites | 2301.000 | 1501.000 | 12478.000 | 144.000 | 2920.082 |
Journal Impact Factor | 2.540 | 2.380 | 8.649 | 0.500 | 1.915 |
Impact Factor without Journal Self Cites | 2.249 | 1.902 | 7.108 | 0.390 | 1.703 |
5-Year Impact Factor | 2.950 | 2.440 | 9.243 | 0.406 | 2.236 |
Immediacy Index | 0.430 | 0.426 | 1.390 | 0.053 | 0.336 |
Cited Half-Life | 6.241 | 6.550 | 10.000 | 2.300 | 1.867 |
Citing Half-life | 7.991 | 8.000 | 10.000 | 4.900 | 1.041 |
Eigenfactor Score | 0.003 | 0.002 | 0.013 | 0.000 | 0.003 |
Article Influence Score | 0.752 | 0.481 | 2.792 | 0.091 | 0.682 |
All indicators must be standardized during the evaluation. The normalization method of the forward indicator is to divide the original indicator value by its maximum value. The cited half-life and the citing half-life are two reverse indexes that need to be processed in a forward direction. The standardized formula is as follows:
In formula (
First of all, the TOPSIS method is used for evaluation. For the sake of simplicity, the weights of the nine evaluation indexes are set equal to 0.111. Next, the coordinated TOPSIS is used to evaluate. The weight of the coordination degree is set to 0.5, and the weight of the TOPSIS evaluation is also 0.5. The evaluation results are shown in Table
Comparison of TOPSIS and coordinated TOPSIS evaluation.
JCR Abbreviated Title | Angle | Coordination | TOPSIS | TOPSIS | Coordinated | Ranking |
---|---|---|---|---|---|---|
INT J ROBOT RES | 71.479 | 0.206 | 0.610 | 2 | 0.967 | 1 |
SOFT ROBOT | 72.970 | 0.189 | 0.633 | 1 | 0.947 | 2 |
IEEE ROBOT AUTOM MAG | 72.314 | 0.197 | 0.507 | 4 | 0.864 | 3 |
IEEE T ROBOT | 73.132 | 0.187 | 0.515 | 3 | 0.849 | 4 |
J FIELD ROBOT | 71.539 | 0.205 | 0.426 | 5 | 0.821 | 5 |
BIOINSPIR BIOMIM | 71.751 | 0.203 | 0.415 | 6 | 0.806 | 6 |
ROBOT CIM-INT MANUF | 72.060 | 0.199 | 0.401 | 7 | 0.787 | 7 |
AUTON ROBOT | 70.934 | 0.212 | 0.346 | 8 | 0.773 | 8 |
ROBOT AUTON SYST | 71.140 | 0.210 | 0.302 | 13 | 0.733 | 9 |
SWARM INTELL-US | 72.320 | 0.196 | 0.322 | 12 | 0.718 | 10 |
J BIONIC ENG | 73.465 | 0.184 | 0.333 | 9 | 0.696 | 11 |
J MECH ROBOT | 73.416 | 0.184 | 0.328 | 11 | 0.694 | 12 |
INT J SOC ROBOT | 73.853 | 0.179 | 0.329 | 10 | 0.683 | 13 |
J INTELL ROBOT SYST | 73.478 | 0.184 | 0.278 | 15 | 0.652 | 14 |
ROBOTICA | 72.311 | 0.197 | 0.167 | 20 | 0.596 | 15 |
INT J ADV ROBOT SYST | 76.305 | 0.152 | 0.296 | 14 | 0.593 | 16 |
ADV ROBOTICS | 73.395 | 0.185 | 0.193 | 18 | 0.588 | 17 |
INT J HUM ROBOT | 73.707 | 0.181 | 0.162 | 21 | 0.555 | 18 |
IND ROBOT | 74.480 | 0.172 | 0.171 | 19 | 0.542 | 19 |
APPL BIONICS BIOMECH | 75.557 | 0.160 | 0.159 | 22 | 0.504 | 20 |
INT J ROBOT AUTOM | 77.649 | 0.137 | 0.203 | 16 | 0.484 | 21 |
REV IBEROAM AUTOM IN | 80.377 | 0.107 | 0.195 | 17 | 0.407 | 22 |
Average | 73.529 | 0.183 | 0.331 | - - | 0.694 | - - |
From the average of all the journals, the overall
From the ranking of evaluation results, after considering the coordination degree, the ranking of the evaluation results of the coordinated TOPSIS differs greatly from that of the TOPSIS. There are only four types of journals that are completely consistent in the ranking of the two evaluation methods, accounting for 18.18% of all journals. Because the overall number of robotics journals is small, this situation will be more serious for other disciplines that have more journals.
The degree of correlation between TOPSIS and coordinated TOPSIS evaluation results is relatively high. The correlation coefficient is 0.939, indicating a high degree of consistency between the two. From the scatter plot of the evaluation results (Figure
Comparison of scatter plots of evaluation results.
Coordinated TOPSIS is an evaluation method that takes into account the coordination of evaluation indicators and the advantages of TOPSIS evaluation. In accordance with TOPSIS evaluation, this paper proposes a method to determine the degree of coordination. The principle is to measure the angle between the line from the evaluation object to the origin and the ideal solution to the origin. The smaller the angle is, the better the coordination of the indicators is. Based on this result, a new evaluation method, coordinated TOPSIS, is proposed, combining TOPSIS and coordination level. The weight of the coordination degree can be manually adjusted according to the purpose of evaluation, and it can be widely applied when the coordination level of indicators need to be taken into account
The empirical research shows that the TOPSIS evaluation results have a high correlation with the coordinated TOPSIS evaluation results, but they differ significantly from the ranking perspective. Because there are few robot academic journals in this study, this sort of difference has been shown. The ranking difference will be even greater for subjects with more academic journals. The evaluation method cannot be simply chosen for a higher correlation coefficient. It should be based on the purpose of the evaluation and the principle of the evaluation method. After all, the choice of evaluation methods will have a significant impact on the results.
Coordinated TOPSIS is very suitable for evaluation field that needs to consider the coordinated development, and it can be replicated. In the evaluation of science and technology, the balanced development of evaluation indicators has become more and more important, such as academic quality and influence, technology and economy, scientific R&D, and transformation of achievements, etc. As far as the field of economic and social development is concerned, more areas need to consider coordinated development. Therefore, coordinated TOPSIS has a wide range of applications, which can be selected according to the purpose of evaluation and extended to some extent.
The data used to support the findings of this study are included within the article.
The authors declare that they have no conflicts of interest.
This paper is supported by the Major Project of National Social Science Fund of China (No. 16ZDA053), Humanities and Social Sciences Projects of the Ministry of Education (17YJA630125), Philosophy and Social Science Foundation of Zhejiang Province (17NDJC107YB), and China’s National Natural Science Foundation Project (71663058). The authors thank them heartedly for supporting the paper funds.