Multilevel Evaluation of Teaching Quality in Higher Education Using Single-Valued Neutrosophic Set

Evaluation of teaching quality is essential for teachers’ promotion, students’ course selection, and institutes’ standing. A multilevel evaluation framework for teaching quality in higher education was investigated by combining the Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) with the single-valued neutrosophic set (SVNS). An indicator system was constructed, including the teaching performance and the students’ learning outcomes. For the qualitative indicator values, an SVNS representation method was proposed, aiming to describe the uncertainty and improve the credibility and validity of the evaluation. Then, both the qualitative data and quantitative data were applied to the TOPSIS-based multilevel evaluation framework, which consisted of an overall assessment and five specific evaluations. The former assessment would provide a final rank and determine the best lecturers to be given priority in awards and promotion. The latter would focus on identifying areas where the lecturers could do better and would give them tips to overcome their challenges. Finally, a descriptive example was provided to verify the proposed framework and demonstrate its practicality.


Introduction
Assessment of teaching quality in higher education is a topic of discussion worldwide. An accurate evaluation of teaching quality can be beneficial to decision-making and help promote educational outcomes. It also has a significant impact on employment, tenure, and promotion of lecturers. However, presently the process of evaluation is complex and challenging, and there are no widely accepted methods [1]. Additional efforts are required to find better ways to address this problem. Spooren et al. provided an extensive review of the high impact studies [2].The construction of the assessment instrument is a significant point of discussion. Baghdadi attempted to achieve the betterment of teaching quality by optimizing the choice of valid indicators to evaluate the teaching quality. Based on the analysis of an exploratory questionnaire, a new indicator system was proposed, which included four factors. It was claimed that the new indicators would enable students to evaluate teaching in terms of their learning progress [3]. Matosas-López et al. analyzed the lim-itations of the current evaluation systems in assessing the university teachers in blended learning modalities and constructed an assessment indicator system with behavioral scales. The proposed indicator system highlighted the teaching aspects of blended learning models and reinforced the formative purpose of assessments [4].
Another important issue is the methods and tools employed for efficient assessment. Nurdin et al. reported a new way to evaluate teaching quality using an online mobile system. An android system was developed and applied in an Islamic higher education institution. The results showed that the online mobile system could increase the participation rate in the assessment of teaching quality [5]. Esmael refuted the unreliability of the online teaching evaluation by investigating the difference between the manual course evaluation and online assessment. Totally, 4678 students and 180 courses were utilized for the assessments. The results showed no significant difference between the feedback received on the manual and online assessments; therefore, the use of the online teaching evaluation continued to be encouraged [6]. Literature [7] focused on the problem of assessing teaching quality in the massive open online courses (MOOC) modality. The authors construed a Hadoop platform and developed a video player to collect and calculate the learning time. Based on these, a naive Bayesian model was designed and applied to the collected data. Finally, the learning cost coefficient was obtained to describe the teaching quality.
Although previous works have achieved success in various applications, we found that there were some issues worthy of in-depth research. First, most existing methods put too much emphasis on teachers' attributes, such as their academic level, teaching ability, and teaching skills. The students' learning outcomes were often excluded from those indicators that were used to evaluate teaching quality [8]. As a result, the evaluation results were not essentially credible, and high teaching quality was not intrinsically related to higher learning outcomes. Outcome-based education (OBE), put forward by Spady [9], is a new and widely used education model. The changes introduced by OBE emphasized what the students learned rather than what they were taught. Everything in an educational system should be organized to ensure that students achieve the expected goals [10]. According to this theory, the students' learning outcomes should be considered in evaluating the teaching quality [11].
Another significant issue was the method to deal with the qualitative indicator values. Unlike quantitative data, qualitative data often mirror the pattern of how people think and make judgments. Linguistic variables are usually employed to express qualitative data rather than numbers. The values are based on opinions, feelings, or viewpoints. There exists much incomplete and uncertain information [12]. During the past decades, many approaches have been investigated to deal with the qualitative evaluation results, aiming to improve the validity and credibility of teaching quality evaluation [13]. Among them, fuzzy-based methods seem to be an appealing tool. It has been widely accepted that reasoning based on fuzzy approaches provides an attractive way to deal with imprecise data [14]. Motivated by this, Li established a fuzzy model to evaluate the quality of teaching English in colleges, based on the analytic hierarchy process (AHP) method and Grey system theory. The author claimed the model could effectively assess the quality of English teaching, and the evaluation results, in turn, greatly promoted the quality of teaching English in colleges [15]. Similarly, a framework based on the multiattribute fuzzy measurement model was investigated to assess the teaching quality of DanceSport Major. The model was also believed to offer an excellent assurance to improve the teaching quality of DanceSport Major [16]. Recently, neutrosophic set (NS), defined as the generalization of interval fuzzy sets, has been used extensively in many fields, such as medical image processing, decision-making, and social judgments. It has been widely accepted that the NS-based technique supplies an effective tool for handling inconsistent and vague data [17][18][19][20][21]. Motivated by this, NS was increasingly employed to represent the qualitative indicator values in various fields, aiming for improving the credibility of the applications.
The last concern was the use of the evaluation result. Most of the existing studies provided only the final rank of the teaching quality, for each lecturer. In literature [22], the authors combined the Grey correlation with the TOPSIS to provide a modified Grey-TOPSIS method and applied it to evaluate the teaching quality in five colleges. The main disadvantage was that the evaluation only provided a final rank for the five colleges. The evaluated colleges received no valuable feedback and could not benefit from the assessment. We believe that if the problems causing the poor ranks could be identified, the evaluation would be more beneficial. Therefore, we struggled to propose a multilevel evaluation framework, which not only provided the final grade but also identified problems and provided timely feedback for each lecturer.
The main contributions of this study are as follows: (1) an evaluation indicator system was constructed, which covers the teaching performance and the students' learning outcomes and provides a strong foundation for efficient assessment. (2) The qualitative indicator values were analyzed from the SVNS perspective, which would be useful to describe the uncertainty of the qualitative data and to improve the credibility and validity of the evaluation. (3) A multilevel evaluation framework of teaching quality in higher education was proposed, which could not only provide a final rank for the lecturers but also timely feedback, which would be beneficial to improve their teaching activities.
The remainder of this paper is organized as follows: Section 2 provides a brief introduction of the techniques utilized in this study. Section 3 discusses the multilevel evaluation method using SVNS and TOPSIS. Section 4 discusses a typical case to illustrate how to apply the evaluation approach followed by the conclusion in Section 5.

Preliminaries
2.1. Definition of NS. NS was proposed by Smarandache [18]. Each element of NS has a degree of truth, indeterminacy, and falsity, which are independent. Let X denote a universe of discourse and x ∈ X. The NS A can be defined by a truth membership function T A ðxÞ, an indeterminacy membership function I A ðxÞ, and a falsity membership function F A ðxÞ. The definition can be depicted as follows: T A ðxÞ, I A ðxÞ, and F A ðxÞ are real standard or real nonstandard subsets of 0 − , 1 + ½, i.e., There exist no restrictions on the sum of the functions of T A ðxÞ, I A ðxÞ, and F A ðxÞ; therefore, where sup ð•Þ indicates the supremum operation of a set.  [23]. The interval [0, 1] was used instead of 0 − , 1 + ½ to achieve better representation and application to practical issues. The SVNS B can be constructed as follows: where T B x ð Þ: X ⟶ 0, 1 ½ , And the limitation of the three membership functions changes to be The systematical operational rule for SVNS can be found in literature [20,21]. In this study, we focused on the application of SVNS in processing the evaluation data of teaching quality.

TOPSIS Ranking Method. TOPSIS was introduced by
Hwang and Yoon. According to this technique, the ideal point denotes the one consisting of all the best indicator values possible, and the point that consists of all the worst indicator values possible is called the negative-ideal point. The solution, which approaches the ideal point and is distant from the negative-ideal point, is usually ranked as a better one [24].
Let I + denote the ideal point and I − represent the negative-ideal point. The closeness coefficient of the solution

Mobile Information Systems
k, which can be used to describe its superiority, is defined in Equation (7). The alternatives having a higher closeness coefficient would be considered as the superior ones.
where WD + k is the distance from the solution k to the ideal point I + and WD − k denotes the distance from the solution k to the negative-ideal point I − .

Methodology
3.1. Indicator Selection. The selected indicators for the evaluation of teaching quality are shown in Table 1, which shows that this indicator system covered the aspects regarding teaching performance and those concerning the students' learning outcomes. This would make the evaluation complete and provide more information on the teaching quality.
In addition, scientific methods were also developed to collect indicator values. For the quantitative indicators, the assessment data were imported from the Center for Teaching Quality Assessment of the university. We believe that this is the most authoritative way to collect the indicator values. For the qualitative ones, questionnaires were employed to collect the assessment of experts and students. The questionnaires contained all the qualitative indicators listed in Table 1. Nine answer options were provided for each item. These were as follows: extremely good/high, very good/high, good/high, medium good/high, medium/fair, medium bad/low, bad/low, very bad/low, and extremely bad/low. Experts and the students who took the evaluated courses would be invited to make their choices.

Tendency Treatment.
According to the principle of TOPSIS, the values of the cost indicators were first transformed into beneficial values. In this study, the original values of the cost indicators C 1 , C 2 , and C 3 were applied to Equation (8), and the corresponding benefit values were obtained.
where C k i denotes the original value of the indicator C i for the lecturer k and C k,P i is the obtained beneficial value of C k i .

Normalization for Quantitative Data.
To avoid comparing indicator values on different scales, all the quantitative data are normalized according to Equation (9).
where NC k i is the normalization result of the original value C k i . 3.2.3. Quantification for Qualitative Data. The input data, expressed in a nonnumerical way, was quantified through the SVNS representation technique. Table 2 shows the SVNS representation for the fuzzy linguistic variables.

Mobile Information Systems
Suppose there were T evaluators (including experts and students) providing their feedback for each lecturer regarding each qualitative indicator. The SVNS weight aggregation operator was utilized to aggregate the different representations given by different evaluators according to the given weights λ. Equation (10) illustrates the aggregation operation.
where FC k,t i is the SVNS representation of the indicator C i for the k th lecturer provided by the t th (t = 1, 2, ⋯, T) evaluator. FC k i denotes the aggregated indicator value.

Multilevel Evaluation Framework.
In this study, a multilevel evaluation framework was proposed using TOPSIS technique. It consisted of an overall evaluation and five specific evaluations, as shown in Figure 1.. The overall evaluation was performed using all the indicators listed in Table 1 and provided a credible rank for each evaluated lecturer. The specific evaluations only used the particular indicators of one aspect. Their functions were to identify areas for improvement and individualizing professional development.

Mobile Information Systems
Suppose there were M quantitative indicators involved in an evaluation. Let DI + denote the quantitative subset of the ideal point I + and DI − represent the quantitative subset of the negative-ideal point I − . The definition of DI + and DI − were described as Equations (11) and (12).
Correspondingly, the quantitative parts of the distances WD + k and WD − k can be expressed as Equations (13) and (14).
where w m is the weight for the m th indicator and can be determined using the method of the SVNS cross-entropy [25]. Similarly, suppose there were N qualitative indicators involved in the evaluation. FI + and FI − were the qualitative subsets of I + and I − , respectively. Their definitions were as Equations (15) and (16).
The qualitative part of the distances WD + k and WD − k can be expressed as Equations (17) and (18).
where w n is the weight for the n th indicator and can be determined in the same way as w m in Equations (13) and (14).
Let w d and w f be the weights of the quantitative part and the qualitative aspect of the distances, subject to 0 ≤ w d , w f ≤ 1, w d + w f = 1. The comprehensive weighted distances WD + k and WD − k could be obtained according to Equations (19) and (20). If only quantitative indicators were involved in the evaluation, w d equaled to 1 and w f was 0. In contrast, when the assessment only relied on the qualitative indicators, w d was set as 0 and w f was 1.
Finally, the closeness coefficient can be obtained according to Equation (7).

An Example of Teaching Quality Evaluation
The applicability of the proposed framework was illustrated with the help of an example, followed by a discussion based on the evaluation results. Eighty-four lecturers were involved in this evaluation. The Center for Teaching Quality Assess-ment provided the quantitative data for the lectures. Twenty experts and all the students who took the related courses were invited to make their qualitative assessment of the lecturers. While transforming the qualitative assessments made by the experts and students into SVNS representations, the weight λ for each expert was set as 0.4 and the weight for each student evaluator was set as 0.6. When calculating the final closeness coefficient, the weights w d and w f were set as 0.3 and 0.7, respectively. The evaluation was performed on a self-developed software. Figure 2 shows four typical interfaces of the software. Figure 3(a) shows the overall evaluation results of the 84 lecturers. Most lecturers' closeness coefficient fluctuates around 0.5, indicating that the settings of the ideal solution and the negative ideal solution are reasonable. The five lecturers of No. 25,No. 35,No. 49,No. 66,and No. 82 have high closeness coefficients, which are all greater than 0.7. It means that the teaching quality of these five lecturers is the best among the 84 lecturers. The five lecturers were requested to share their practices throughout the university and were recommended to be given priority in awards and promotion. Figures 3(b)-3(f) illustrate the results of the five specific evaluations. It was observed that there were 49 lecturers who had closeness coefficients in at least one specific evaluation Mobile Information Systems lower than the preset threshold, which equaled to 0.2. These lecturers would receive a written feedback and needed to participate in mentoring sessions on teaching skills. The instructional coach would review the evaluation feedback with them and help them identify the areas of improvement in their own teaching skills. Thereafter, a data-driven action plan would be created for improving the 49 lecturers' teaching practices. If a lecturer achieved low scores in two consecutive semesters, they would be required to suspend teaching for a short period. Only after passing an additional examination could they restart their teaching practice.

Conclusion
Teaching quality is an intangible concept, and there is an increasing interest in its evaluation methods. This study demonstrates that SVNS, combined with TOPSIS, provides a novel tool in addressing this problem. A case study was also provided to demonstrate how to employ the method to assess the teaching quality. The teaching quality of 84 lecturers was evaluated with the quantitative and qualitative data collected. As expected, the overall evaluation clearly indicated the five lecturers who achieved the best teaching performance. Additionally, the five specific evaluations identified the issues related to lecturers' growth areas that they might need to address. That would help them improve their teaching practice. However, the weights, such as λ in Equation (10) and ω in Equations (17)- (20), were specified by experts before the beginning of the evaluation in this study. It is a simple method, but the values may not be the optimal one. Further efforts are suggested to find the best way to optimize the value of the weights. In addition, the proposed method needs to be tested on more teaching practices from different higher education institutions.

Data Availability
The data used to support the findings of this study are available from the corresponding author upon request.

Conflicts of Interest
The authors declare that they have no conflicts of interest.