The purpose of this study is to evaluate the training institution performance and to improve the management of the Manpower Training Project (MTP) administered by the Semiconductor Institute in Taiwan. Much literature assesses the efficiency of an internal training program initiated by a firm, but only little literature studies the efficiency of an external training program led by government. In the study, a hybrid solution of ICA-DEA and ICA-MPI is developed for measuring the efficiency and the productivity growth of each training institution over the period. The technical efficiency change, the technological change, pure technical efficiency change, scale efficiency change, and the total factor productivity change were evaluated according to five inputs and two outputs. According to the results of the study, the training institutions can be classified by their efficiency successfully and the guidelines for the optimal level of input resources can be obtained for each inefficient training institution. The Semiconductor Institute in Taiwan can allocate budget more appropriately and establish withdrawal mechanisms for inefficient training institutions.
Due to the fast growth and expansion of high-tech industry in Taiwan, high-tech companies in the semiconductor industry have a strong demand for technical and R&D talents, but the academic institutions, the major supplier of high-tech manpower, cannot train enough students to fulfill this demand. According to an investigation conducted by Taiwan’s Science and Technology Advisory Group of Executive Yuan in 2001, the technical manpower shortage for the semiconductor industry alone was about 6,600 people during 2003 to 2005. Facing severe global competition, both Taiwan authorities and the industry must find another way to cultivate more high-tech talents efficiently to pursue sustainable development and to maintain competitive advantage. Therefore, the Industrial Development Bureau (IDB) at the Ministry of Economic Affairs established the Semiconductor Institute to implement the Manpower Training Project (MTP) for matching up with the “Challenge 2008 National Development Plan” of Executive Yuan in 2003.
The main objective of the MTP is to fulfill the shortage of semiconductor manpower by providing training classes to those who want to pursue a career in the semiconductor industry. The career training program was carried out by various training institutions, which are affiliated with university/college, research institutions, or grassroots organizations, in northern, central, and southern Taiwan. During the year of 2003 to 2005, MTP’s career training program has teamed up with over 20 training institutions to provide 283 classes with different training lengths. 3,950 graduates of the MTP have made contributions to the semiconductor industry.
Even though the MTP has been implemented for years and the success of the MTP has been evidenced by the accumulated number of trainees since 2003, the Semiconductor Institute encounters several challenges in project management. First, no scientific evaluation method has been established for measuring the project implementation efficiency of training institutions which are responsible for course design, student enrollment, and student job replacement. Second, no official withdrawal mechanism for inefficient training institutions and an optimal resource allocation are developed for improving the overall performance of the Manpower Training Project (MTP).
The issues faced by the Semiconductor Institute are not unusual for project management in general. Like the MPT, a project could be administered by a project manager in one organization (like the Semiconductor Institute), but the implementation of this project is assigned to multiple decision-making units (like the training institutes) with the same objective, but different competence and execution ability. Decision-making units receive the guidance and financial support from the project manager, and they will be evaluated by the project manager at the end of the project execution period as well. The example includes the project management in the health care and in the financial service industry.
To evaluate the performance of decision-making units, various efficiency measurement tools, such as conventional statistical methods, nonparametric methods, and artificial intelligence methods, have been successfully developed in the literature. Among these tools, the data envelopment analysis (DEA) approach has received the most discussion. DEA is known as the efficient frontier approach [
Even though DEA has been applied in efficiency measurement successfully, the presence of strong correlation among the input variables of a DMU can bias the efficiency estimates of a DMU in the slack analysis [
In this study, to evaluate the training institution performance administered by the Semiconductor Institute in Taiwan, we first applied the PCA-ICA technique proposed by Kao et al. [
The proposed solution is illustrated by a dataset provided by the Semiconductor Institute in Taiwan. The dataset contains the input and output information of ten training institutions which joined the Semiconductor Institute’s Manpower Training Project in 2009 and 2010. Five input variables required to deliver the training course include the total number of professional-qualified faculty, the total number of academic-qualified faculty, the total number of administrative staffs, the average practical training hours, and the total number of graduates who majored in semiconductor-unrelated fields. Two project outputs are the total number of successful employment placements and the total number of graduates from the training institution.
In the empirical study, we compared the outcomes of ICA-DEA and single DEA to demonstrate the influence of correlated input variables on the discrimination capability of DEA. Our analysis shows that ICA-DEA approach can avoid efficiency misjudgment. They are consistent with the conclusion made by Kao et al. [
This paper contributes to the literature and the semiconductor industry in two aspects. First, the proposed hybrid approach of ICA-DEA and ICA-MPI can be applied to study the efficiency of DMUs and help the project manager to manage his project more appropriately (e.g., provide improvement suggestion). The proposed approach can also be extended to other similar applications in business. For example, a firm’s HR department can use the proposed approach to evaluate the efficiency of a HR training project in multiple campuses. Second, the proposed approach can provide guidance of resource allocation in project management. For example, the Semiconductor Institute can set up optimal input levels for each training institution to maximize the number of job replacements and the number of graduates. The project management practice of the Semiconductor Institute can also be extended to other government projects.
The rest of this paper is organized as follows: Section
Most project management literature focus on project management strategies, project planning and control, process improvement, risk management, simulation modeling, leadership and team building, or negotiation and contracting strategy. Due to the temporary nature of a project, not much literature is concerned about project efficiency evaluation and performance enhancement for some projects, like the Manpower Training Project in Taiwan, which is temporary for the project execution organization (training institutions) but is recursive for the organization (the Semiconductor Institute) which administers this project.
However, regardless of industry application, evaluating project performance is critical to any organization because project managers can avoid making similar mistakes from experience. Then if projects with the same properties will be executed again, the entire process of project management can be done much smoothly and efficiently.
In the literature, data envelopment analysis (DEA) has been widely used as a benchmarking approach in evaluating project productivity performance [
Along with DEA’s popularity, DEA’s two drawbacks must be acknowledged. First, when the strong correlation among inputs of a DMU is observed, the result of slack analysis can be biased [
Besides, these applications in the literature are not suitable for managing a project which has the following characteristics. First, the project itself is administered and executed with minor adjustment recursively over a period of time. Second, the project is planned, organized, led, and controlled by one administrative organization but is executed by multiple execution institutions which are not affiliated with the administrative organization. Third, the administrative organization allocates budget to execution institutions periodically, evaluates the performance of each execution institution, and decides if the contract with execution institutions should be renewed. Fourth, the administrative organization can provide improvement suggestion to inefficient execution institutions, but inefficient execution institutions may not necessarily follow the improve suggestions due to some unobservable factors.
Therefore, a new method of project performance evaluation is needed. In this research, we propose a hybrid method which combines ICA, DEA, and MPI together to solve the project performance evaluation problems as illustrated above. In the following section, a brief introduction of each method is provided.
Data envelopment analysis is known as the efficient frontier approach [
DEA and its modification have been increasingly used over the past decade to measure performance. For example, DEA has been applied to evaluate the performance of supply chain [
Along with DEA’s popularity, DEA’s two drawbacks must be acknowledged before it is applied. First, when the strong correlation among inputs of a DMU is observed, the result of slack analysis can be biased. It is because DEA used the weighting method to calculate the ratio between inputs and outputs of each DMU. Second, DEA’s discrimination capability is lessened if the model is misspecified or the number of DMUs is too small.
Even though Adler and Golany [
Productivity is a relative concept which is used to measure, analyze, and monitor a DMU’s project execution ability relative to itself in the past year or to other DMUs at the same year. Even though productivity can be defined in various ways, the Malmquist productivity index (MPI), which was introduced by Malmquist [
Based on multi input-output frontier representations of the production technology [
According to Färe et al. [
Färe et al. [
For “Technology Change (Tech-Ch)” in (
Among various methods used to measure the distance functions, which make up MPI, the DEA-like method [
ICA can be viewed as an extension of principal component analysis (PCA) with a different objective [
ICA is a methodology for capturing both second and higher order statistics, and it projects the input data onto the basis vectors that are as statistically independent as possible [
The literature has applied ICA in human face recognition on FERET database [
For illustrative purpose, we can assume that each of
Typically, the statistical independence of ICs can be measured in terms of their non-Gaussian properties [
The schematic representation of the proposed model is illustrated in Figure
The research scheme of the proposed analysis model.
In the second stage, ICA is applied to the input variables of TIs in 2009 and 2010 for generating ICs simultaneously. The estimated ICs, regarded as the key factors affecting productivity growth, are then utilized as new input variables in the Malmquist productivity index (MPI) to see if the productivity of inefficient TIs in 2009 could apparently grow in the following year (2010) after the suggested slack entries are considered. Finally, Semiconductor Institute could consider reallocating budget for each TI based on its corresponding efficiency and withdrawing those inefficient TIs that fail to improve in 2010.
In this study, the dataset of training institutions in 2009 and 2010 provided by the Semiconductor Institute in Taiwan is used to illustrate the proposed ICA-DEA and ICA-MPI approaches. The data contains the information of ten training institutions which joined the Semiconductor Institute’s Manpower Training Project in 2009 and 2010. According to their functional complexity, these ten training institutions can be categorized into three different classes: (1) universities and colleges; (2) research institutions; and (3) grassroots organizations. Among these ten training institutions, six are located in northern Taiwan while four are located in southern Taiwan.
Because, regardless of methodologies, the result of efficiency measurement is highly influenced by the selection of input and output variables, reviewing the literature for variable selection in similar studies is needed. We found that, in the literature, the DEA has been applied in studying school efficiency [
Definition and explanation of variables.
Variables | Definition and explanation | |
---|---|---|
Inputs | Industry_faculty ( |
The total number of professionally qualified faculty from the industry within a training institution |
Academic_faculty ( |
The total number of academically qualified faculty from universities within a training institution | |
Administrative_staffs ( |
The total number of administrative staffs employed in a training institution | |
Project_hours ( |
The average hours of the course project which each student spent in practical training | |
Student_# in unrelated field ( |
The total number of graduates who majored in semiconductor-unrelated fields | |
|
||
Outputs | Successful employment ( |
The total number of successful employment placements |
Trainee_# ( |
The total number of trainees graduated from a training institution |
Rescaled input and output variables and their summary statistics.
Year | Training institution |
|
|
|
|
|
|
|
---|---|---|---|---|---|---|---|---|
2009 | 1 | 0.2300 | 0.0496 | 0.4720 | 0.0100 | 0.1600 | 0.0100 | 0.1623 |
2 | 0.4500 | 0.5644 | 1.0000 | 0.5417 | 1.0000 | 0.8122 | 0.9086 | |
3 | 0.1200 | 0.2080 | 0.4940 | 0.2025 | 0.1000 | 0.0100 | 0.0252 | |
4 | 0.6700 | 0.3268 | 0.1640 | 0.8350 | 0.0700 | 0.1978 | 0.1775 | |
5 | 0.7800 | 0.3268 | 0.4500 | 0.6150 | 0.2200 | 0.3855 | 0.2689 | |
6 | 0.8900 | 0.0100 | 0.5380 | 0.6150 | 0.6700 | 0.1466 | 0.0709 | |
7 | 0.6700 | 0.1684 | 0.4940 | 0.5233 | 0.3700 | 1.0000 | 1.0000 | |
8 | 0.3400 | 0.4456 | 0.2080 | 0.3033 | 0.1300 | 0.7269 | 0.6040 | |
9 | 0.5600 | 0.2080 | 0.4280 | 0.5508 | 0.3400 | 0.3002 | 0.3298 | |
10 | 0.2300 | 0.5644 | 0.3620 | 0.3858 | 0.4000 | 0.2831 | 0.3451 | |
|
||||||||
2010 | 1 | 0.0100 | 0.2476 | 0.0100 | 0.1567 | 0.0700 | 0.1978 | 0.4669 |
2 | 1.0000 | 1.0000 | 0.8900 | 1.0000 | 0.9400 | 0.7952 | 0.7715 | |
3 | 0.8900 | 0.6832 | 0.4940 | 0.8442 | 0.3100 | 0.0441 | 0.0100 | |
4 | 0.1200 | 0.1684 | 0.0760 | 0.1108 | 0.0100 | 0.1466 | 0.3755 | |
5 | 1.0000 | 0.1684 | 0.4060 | 1.1192 | 0.4000 | 0.5050 | 0.4822 | |
6 | 0.6700 | 0.1288 | 0.4720 | 0.5050 | 0.0700 | 0.1124 | 0.0862 | |
7 | 0.5600 | 0.3664 | 0.4060 | 0.6608 | 0.2500 | 0.9488 | 0.9238 | |
8 | 0.5600 | 0.2872 | 0.1420 | 0.3400 | 0.2800 | 0.3855 | 0.3908 | |
9 | 0.2300 | 0.0892 | 0.3400 | 0.3950 | 0.4300 | 0.9488 | 0.8020 | |
10 | 0.1200 | 0.6436 | 0.2960 | 0.1475 | 0.1900 | 0.3172 | 0.2842 | |
|
||||||||
Mean | 0.5050 | 0.3327 | 0.4071 | 0.4931 | 0.3205 | 0.4137 | 0.4243 | |
Std. Dev. | 0.3162 | 0.2505 | 0.2396 | 0.3018 | 0.2738 | 0.3370 | 0.3144 |
Practically, subsets of the inputs or outputs are always correlated. The high degree of correlations between the variables could cause issues with the distribution of the weights. Dropping a highly correlated variable from the assessment could not only reduce the efficiency ratings for some DMUs [
The correlation matrix of input and output variables in our study is provided in Table
Correlation coefficients between variables.
|
|
|
|
|
|
| |
---|---|---|---|---|---|---|---|
|
1 | — | — | — | — | — | — |
|
0.1809 | 1 | — | — | — | — | — |
|
0.4531 | 0.3849 | 1 | — | — | — | — |
|
0.8936 | 0.3203 | 0.4205 | 1 | — | — | — |
|
0.4464 | 0.4284 | 0.8159 | 0.4703 | 1 | — | — |
|
0.2926 | 0.4754 | 0.4638 | 0.4215 | 0.6043 | 1 | — |
|
0.3627 | 0.3995 | 0.5907 | 0.4463 | 0.7508 | 0.9475 | 1 |
We used the BCC input-oriented model in the
For the proposed ICA-DEA model, we first applied the basic ICA approach to estimate a de-mixing matrix
The results of efficiency measurement for both single DEA method and ICA-DEA method are reported in Table
Summary of the results of single DEA and ICA-DEA models in 2009.
Year | Single DEA model | ICA-DEA model |
---|---|---|
Average score | 0.9492 | 0.7514 |
Standard deviation | 0.0859 | 0.2682 |
Maximum efficiency score | 1 | 1 |
Minimum efficiency score | 0.7778 | 0.3875 |
Number of efficient DMUs | 7 | 5 |
Total number of DMUs | 10 | 10 |
Percentage of efficient DMUs | 70 | 50 |
DEA provides not only the efficiency results but also slack analysis, by which guidelines for the optimal level of input and output resources can be derived for each training institution. That is, each training institution could have its input and output resources set at the optimal level—the original level minus the inefficient and slack amounts from the DEA results [
Table
Slack analysis of ICA-DEA method for input variables.
Training institution | Industry_faculty ( |
Academic_faculty ( |
Admin_staffs ( |
Project_hours ( |
Student_# in unrelated field ( |
---|---|---|---|---|---|
1* | 0 | 0 | 0 | 0 | 0 |
2 | −1 | −6 | −20 | −12 | −15 |
3 | 0 | −3 | −5 | −15 | −3 |
4 | −2 | −3 | −4 | −35 | −2 |
5* | 0 | 0 | 0 | 0 | 0 |
6 | 0 | −3 | −3 | −15 | −5 |
7* | 0 | 0 | 0 | 0 | 0 |
8* | 0 | 0 | 0 | 0 | 0 |
9* | 0 | 0 | 0 | 0 | 0 |
10 | −6 | −2 | −6 | −52 | −7 |
Table
The Malmquist productivity indices explained in Section
Malmquist indices of training institution.
Training institution | Technological change (Tech-ch) | Technical efficiency change |
Pure technical efficiency change |
Scale efficiency change (Se-ch) | Total factor productivity change |
---|---|---|---|---|---|
1* | 1.732 | 1.000 | 1.000 | 1.000 | 1.732 |
2 | 1.178 | 1.000 | 1.000 | 1.000 | 1.178 |
3 | 0.846 | 0.909 | 1.562 | 0.582 | 0.769 |
4 | 1.740 | 1.200 | 1.077 | 1.114 | 2.088 |
5* | 1.360 | 0.821 | 0.687 | 1.195 | 1.116 |
6 | 1.151 | 0.809 | 1.185 | 0.682 | 0.931 |
7* | 1.231 | 0.835 | 1.000 | 0.835 | 1.028 |
8* | 1.326 | 1.040 | 1.000 | 1.040 | 1.379 |
9* | 1.014 | 1.000 | 1.000 | 1.000 | 1.014 |
10 | 1.681 | 1.340 | 1.000 | 1.340 | 2.253 |
|
|||||
Average | 1.293 | 0.983 | 1.032 | 0.953 | 1.271 |
Malmquist index summary of studied training institution.
As shown in Table
Table
When the values of “Eff-ch” are examined, the 10th training institution had the largest increase in “Eff-ch” with a rate of 34.0% while the other training institutions were almost detected with a decrease. The “Eff-ch” values of the 6th, 5th, 7th, and 3rd training institution were decreased by 19.1%, 17.9%, 16.5%, and 9.1%, respectively. For “Pe-ch,” the 3rd, 6th, and 4th training institutions show an increase of 56.2%, 18.5%, and 7.7%, respectively, while the 5th training institution had a decrease of 31.3%. There is no change seen in the “Pe-ch” values of other training institutions during the same period of time.
For the values of “Se-ch,” the 10th training institution had the largest increase with a rate of 34.0%. The “Se-ch” values of the 5th, 4th, and 8th training institutions were increased by 19.5%, 11.4%, and 4.0%, respectively. This situation reveals that the training institution had gained success in means of production realized via appropriate scale adjustment.
Table
In summary, the following observations can be reached by MPI. The 4th and 10th training institutions have the highest values of “Tech-ch” while the 3rd training institution has the lowest value of “Tech-ch.” Thus, it is concluded that the 4th and 10th training institutions had gained success in catching up the production limits. The 10th training institution has the highest “Eff-ch” value, and the 6th training institution has the lowest “Eff-ch” value. The 3rd training institution has the highest “Pe-ch” value while the 5th training institution has the lowest “Pe-ch” value. The 10th training institution has the highest “Se-ch” value while the 3rd training institution has the lowest “Se-ch” value. The 10th training institution has the highest “Tfp-ch” value while the 3rd training institution has the lowest “Tfp-ch” value.
From our analysis, we can conclude that the most successful training institution is the 10th training institution due to its improvement in “Eff-ch,” “Se-ch,” and “Tfp-ch.” And, the 3rd and 4th training institutions are the most successful training institution due to their improvement in “Pe-ch” and “Tech-ch,” respectively. Among these ten training institutions, the 10th training institution is worthy of gaining more budget from Semiconductor Institute because it takes the suggested slack entries from analysis and becomes the most successful training institution. Moreover, the 3rd and 6th training institutions were considered to be withdrawal because they do not meet expected targets during execution.
While much can be accomplished at the project management perspective, several policy implications and conclusions can emerge from this study. All of these recommendations must be considered in light of the context and goals of the project in which they are applied. Thus, the following suggestions are offered.
First, our research suggests that the proposed ICA-DEA method can provide a robust assessment of project performance because the ICA-DEA method has more ability to distinguish between the performances of training institutions and to help the project manager to manage and evaluate his project more appropriately.
Second, the slack analysis of ICA-DEA provides a clear guidance for resource allocation. Our application evidences that the slack analysis of ICA-DEA can enhance the overall productivity of the Manpower Training Project. The Semiconductor Institute in Taiwan can apply the proposed method to evaluate training institutions during their execution period.
Finally, ICA-DEA and ICA-MPI can help a project manager establish a scientific withdrawal mechanism for inefficient DMUs. In our study, Manpower Training Project (MTP) can be executed more effectively and the resources can be allocated more reasonably.
In this research, a hybrid solution of ICA-DEA and ICA-MPI is proposed to evaluate productivity of a project which has the following characteristics. First, the project itself is administered and executed with minor adjustment recursively over a period of time. Second, the project is planned, organized, led, and controlled by one administrative organization but is executed by multiple execution institutions which are not affiliated with the administrative organization. Third, the administrative organization allocates budget to execution institutions periodically, evaluates the performance of each execution institution, and decides if the contract with execution institutions should be renewed. Fourth, the administrative organization can provide improvement suggestion to inefficient execution institutions, but inefficient execution institutions may not necessarily follow its improvement suggestion due to some unobservable factors. The example of this type project includes the project performance evaluation in the health care and in the financial service industry.
In summary, this research contributes to the project management literature in three aspects. First, the proposed hybrid approach of ICA-DEA and ICA-MPI can be applied to study the efficiency of DMUs and help the project manager to manage his project more appropriately (e.g., provide improvement suggestion). Second, the proposed approach can be extended to other similar applications in business. For example, a firm’s HR department can use the proposed approach to evaluate the efficiency of a HR training project in multiple campuses, or a firm can use the proposed approach to evaluate outsourcing performance of various business partners. Finally, the proposed approach can provide guidance of resource allocation in project management.
In the future research, our methodology can be further developed in the following direction. First, due to the data limitation, the empirical study can only demonstrate the proposed hybrid approach in two time periods. The model’s performance assessment for multiple time periods can be considered in the future study. Second, our methodology can be extended to evaluate the qualification of a potential contractor (e.g., training institutions) to implement the project. It can reduce the risk of the Semiconductor Institute. Finally, DEA is a nonparametric approach which cannot deal with the stochastic changes which may affect project efficiency and resource allocation over time. Therefore, the stochastic techniques are suggested to be incorporated into the model development.
The authors declare that there is no conflict of interests regarding the publication of this paper.
This research leading to these results has received funding from the National Science Council in Taiwan under Grant agreement no. NSC 102-2410-H-027-008.