Scientific Programming
Volume 2020 (2020), Article ID 2742437, 7 pages
https://doi.org/10.1155/2020/2742437

Research Article

Selection of Multimode Resource-Constrained Project Scheduling Scheme Based on DEA Method

Song Zhang

School of Management, Xuzhou Medical University, Xuzhou 221004, China

Correspondence should be addressed to Song Zhang; szhang@xzhmu.edu.cn

Received 6 October 2019; Revised 22 March 2020; Accepted 31 July 2020; Published 28 August 2020

Academic Editor: Cristian Mateos

Copyright © 2020 Song Zhang. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract.

A multimode resource-constrained project scheduling problem (MRCPSP) may have multifeasible solutions, due to its nature of targeting multiobjectives. Given the NP-hard MRCPSP and intricate multiobjective algorithms, finding the optimized result among those solutions seems impossible. This paper adopts data envelopment analysis (DEA) to evaluate a series of solutions of an MRCPSP and to find an appropriate choice in an objective way. Our approach is applied to a typical MRCPSP in practice, and the results validate that DEA is an effective and objective method for MRCPSP solution selection.

1. Introduction

Resource-constrained project scheduling problem (RCPSP) has been intensively studied since the 1960s and a standard model for RCPSP has been widely accepted to fulfill scheduling tasks. Despite its preeminence, standard RCPSP can hardly cover all practical situations and many effective extensions have been proposed. Multimode RCPSP (MRCPSP) is one of them. It was first developed by Elmaghraby [1], and it allows multialternative modes that an activity can choose, providing different combinations of resources and durations.

Most of the previous studies focused on solutions to MRCPSP. As Brucker [2] summarized, those algorithms fell into three categories: exact algorithms, heuristics algorithms, and metaheuristic algorithms. Kolisch and Drexl [3], however, proved that whether MRCPSP has a feasible solution is an NP-hard problem when two or more renewable resources and infinite unrenewable ones coexist.

Schnell and Hartl [4] proposed new exact approaches to the MRCPSP with generalized precedence relations. They implemented the optimization framework SCIP which contained two constraint handlers “cumulativemm” and “gprecedencemm” and formulated SCIP models for the MRCPSP. Their solution obtained the outperform results especially on 50 activities when imposing time limits of 27s. Some new metaheuristics have also emerged recently including hybridization of genetic algorithm and fully informed particle swarm [5]; a column generation based distributed scheduling algorithm [6]; a hybrid optimization method which consists of a novel heuristic and unique genetic optimization algorithm for large-scale projects [7]; and an effective mirror-based genetic algorithm [8]. In terms of MRCPSP model extension, some scholars have considered renewable resource vacation and activity splitting [9]. Nemati-Lafmejani et al. [10] proposed an integrated biobjective optimization model to deal with MRCPSP and contractor selection (CS) problem, simultaneously. Hill et al. [11] proposed a reformulation of the waterway ship scheduling problem as a variant of the MRCPSP, which incorporates time-dependent resource capacities besides the earliest and latest start times for the tasks. This problem was solved through integer programming, using a compact mathematical formulation.

Typically, project scheduling achieves ultimate goals in three perspectives: duration, cost, and quality. These multiobjectives make the already complex MRCPSP even harder [12]. Researchers have sought different ways to solve it. Some scholars considered two targets: cost and duration [13]; some tried to maximize net present value and minimize duration [14]; others took all the three (duration, cost, and quality) as objectives to optimize [15].

In practice, duration is usually the most predominant goal of a project, and the priorities of resource, cost, and quality vary with different project executors. Multisolutions to an MRCPSP with different optimized goals may be available and how to evaluate each of them in a subjective way becomes a problem.

Here is a typical scenario. A project manager proposed a number of solutions to a project scheduling, and all their durations met the company’s strategic requirements. Other managers, however, imposed their respective emphases and filed different proposals. For example, CFO focused on capital investment, CHO considered resources, and CMO predicted the quality that would impact the company’s profile. Since all of the solutions were reasonable and feasible, the dilemma that CEO faced then was to evaluate them in an objective and convincible way and make decisions thereafter.

As a matter of fact, the project selection is critical and intricate due to availability of numerous projects, as well as existence of various qualitative and quantitative criteria. The decision-making can involve the enterprise in a long-term commitment. While various methods of project selection have been proposed in the literature, Naldi et al. [16] focus on the profit-fairness trade-off, they adopt a knapsack-type integer linear programming formulation to optimize both quality indicators of budget allocation, employing a fairly wide selection of fairness measures. Liu et al. [17] proposed a novel multiattribute decision-making method based on spherical fuzzy sets for selecting Yunnan Baiyao’s R&D project of toothpastes. However, although the methods mentioned above are effective, they are all targeted at specific types of project selection, which are also complicated and require a lot of information processing in advance.

Furthermore, the most notable among these methods is data envelopment analysis (DEA). DEA is a nonparametric method to measure productive efficiency of decision-making units (DMUs) that involve multi-inputs and multioutputs. This method was first proposed by Charnes et al. [18] in 1978 but rooted in the studies by Debreu [19], Koopmans [20], and Farrell [21] on the measurement of productive efficiency. Thanks to its expertise at performance evaluation, this method has taken great leaps both in theory and in practice. Researchers have applied DEA models to address transportation systems, health care, resource allocation, energy and environmental economics, and regional and national sustainability issues as well as those related to supply chain. Recent overviews of DEA can be found in Mahmoudi et al. [22], Kohl et al. [23], Yang et al. [24], Mardani et al. [25], Zhou et al. [26], Soheilirad et al. [27], Cook and Seiford [28], and Liu et al. [29].

Hitherto, DEA has been applied to solution selections for construction projects [30], and Research and Development (R&D) proposals [31], information project [32, 33], and Six Sigma project [34]. Although, it objectively selected the most efficient projects even under the influences of the executor’s subjective instincts and opinions, there has been little discussion on how to evaluate and select solutions from MRCPSP. There are two major differences. Firstly, the project selection problem deals with opting a set of best feasible proposals from a large pool of projects with making the best use of available resources, while the MRCPSP solution selection is to evaluate different solutions from the same project. Secondly, each project uses different resources and produces various outputs while the latter utilizes the same set of available but limited resources and focuses on the time, cost, quality involved in project management. Our attempt of applying DEA to MRCPSP is to evaluate advantages and disadvantages between multifeasible solutions, providing a solid reference to the decision-maker.

This paper is to investigate the selection of MRCPSP solutions. We adopt data envelopment analysis (DEA) to evaluate available solutions. The contribution of this paper is mainly reflected in two aspects. One is to deepen the application research of MRCPSP and discuss the solution selection from the perspective of relative efficiency. Second, it extends the application field of DEA and the proposed DEA approach could be applied to MRCPSP. Moreover, it provides a new perspective and useful tool for project managers to objectively evaluate a set of similar solutions in an MRCPSP.

The remainder of the paper is organized as follows. Section 2 presents a description of MRCPSP using a virtual case and the solutions of MRCPSP are presented. Section 3 introduces the solution methodology, DEA, designed to solve the problem and computational results are presented and discussed. Finally, conclusions are shown in Section 4.

2. Problem Description and Its Solutions

Suppose there is a project composed of 20 activities, including two dummy ones: the “beginning” and the “termination.” Its activity-on-node (AON) network is depicted in Figure 1. Each activity could run in one of the two modes, the normal and the speedy; the two modes cost differently and lead to different qualities. All the activities could allocate three renewable resources, and their respective totals are 12, 13, and 12. The parameters of the project are listed in Table 1.

Figure 1: An activity-on-node (AON) network of a project.

Table 1: Parameters of a project.

Task
Duration (d)
Cost (yuan)
Quality
Resource

No.
Speedy mode
Normal mode
Speedy mode
Normal mode
Speedy mode
Normal mode
R 1
R 2
R 3

0
0
0
0
0
0
0
12
13
12

1
69
100
110053
91462
0.65
0.95
4
0
0

2
56
89
119303
84874
0.76
0.91
10
0
0

3
64
97
189317
134779
0.48
0.91
0
0
3

4
54
59
13306
8952
0.86
0.91
3
0
0

5
29
44
9244
8492
0.49
0.92
0
6
0

6
69
84
119259
84422
0.55
0.91
0
0
7

7
68
88
189276
134307
0.5
0.89
0
8
0

8
49
59
13262
8500
0.47
1
6
0
0

9
74
94
139445
98523
0.62
0.89
0
0
1

10
66
85
189243
133988
0.76
0.86
0
5
0

11
34
69
88057
44087
0.54
0.91
0
7
0

12
39
59
94600
66178
0.66
0.92
0
0
3

13
24
56
20480
13130
0.66
0.89
0
8
0

14
39
59
90847
63287
0.85
0.89
3
0
0

15
14
33
20458
12904
0.64
0.88
0
0
5

16
39
59
90825
63061
0.55
0.89
0
0
8

17
15
26
20436
12678
0.66
0.88
0
0
7

18
54
74
78571
53749
0.73
0.89
0
1
0

19
9
19
7036
2895
0.6
0.88
0
10
0

20
0
0
0
0
0
0
0
0
0

This is a typical MRCPSP. A project consists of J activities. And two dummy activities are added: the beginning activity 0 and the termination activity J + 1, both of which need zero recourse and zero processing time. Thus, the activities can be defined as a set . Each activity holds M alternative modes. The renewable resources are encapsulated into a set , and the number of resources of type that can be used in each period is described by ; similarly, there is a set of consumable resources and it contains resources of type . , is the set of direct predecessors of activity j, which means activity j cannot begin until every activity is completed. If an activity j processes in mode m, , defines its processing time, the quantity of the renewable resource of type , and the quantity of consumable resource of type .

The ultimate goal is to minimize the total duration of the project, called makespan. Thus, this problem can be described as follows:

subject to

where equation (1) defines the objective function to obtain the minimum processing time; the constraint in equation (2) defines that one activity only processes once in one mode; equation (3) qualifies the timing constraints between activities; equation (4) keeps the number of usable renewable resources in each period less than the total number of the renewable resources; and equation (5) ensures that all the used unrenewable resources cannot exceed the total nonrenewable resources. in these constraints represent the earliest and the latest time windows of an activity, respectively. Table 2 lists all the parameters used in the model.

Table 2: : Parameters used in the proposed model.

Parameter
Meaning

j
Activity number, j = 1, 2, …, J, where J is the total number of activities

The set of all the activities

M
Activity execute modes

k
Resource number, k = 1, 2, …, K, where K is the total number of types of recourses

The type of renewable resources

The type of nonrenewable resources

The number of renewable resources of type k that can be used in each period

The number of nonrenewable resources of type k that can be used in each period

The set of activities immediately preceding activity j

Duration of activity j

The quantity of the renewable resources of type k that activity j requires

The quantity of the nonrenewable resources of type k that activity j requires

The earliest finish time of activity j

The latest finish time of activity j

t
Time number, t = 1, 2, …, T, where T is the deadline of the whole project.

Many scholars proposed novel solutions to MRCPSP, like Van Peteghem and Vanhoucke [35], Wang and Fang [36], Ballestin et al. [37], Liu et al. [38], and Gutjahr [13]. The author of this paper adopted a genetic algorithm, developed by Liu et al., to achieve the ultimate goals. The algorithm was coded by JAVA and run in a PC with double CPUs (Pentium 2.7 GHZ) and a 2.0 GB RAM. And 32 options were selected from the results that provided nondominated solutions within 1000 days, as listed in Table 3.

Table 3: Results of the proposed project by a genetic algorithm.

No.
Duration (d)
Quality
Cost
No.
Duration (d)
Quality
Cost

1
764.3256
0.8273
123.2532
17
739.9256
0.7696
125.6698

2
731.4268
0.8028
128.5642
18
764.723
0.7995
121.6883

3
764.3256
0.8273
123.2532
19
789.7572
0.8268
118.8704

4
731.4268
0.8028
128.5642
20
808.3605
0.8437
118.6849

5
764.3256
0.8273
123.2532
21
858.8132
0.848
113.3372

6
765.7595
0.8283
123.1657
22
713.965
0.7031
133.9032

7
716.4676
0.7845
130.8325
23
738.5133
0.7423
127.0581

8
736.8615
0.805
127.0508
24
780.8691
0.7696
120.6025

9
765.7595
0.8283
123.1657
25
798.7724
0.7971
117.7562

10
808.3605
0.8437
118.6849
26
823.8781
0.8236
115.4423

11
710.9173
0.7682
131.8811
27
855.7369
0.8473
113.5001

12
733.924
0.8003
126.9617
28
897.97
0.8542
112.2478

13
760.7047
0.822
123.3892
29
713.965
0.7031
133.9032

14
783.6034
0.8284
120.4722
30
734.3908
0.7266
128.9113

15
808.3605
0.8437
118.6849
31
766.0741
0.7589
123.6714

16
710.1831
0.7341
131.9792
32
791.3508
0.7735
119.4027

The genetic algorithm eliminated dominated solutions. The results distributions are depicted in Figures 2–4.

Figure 2 demonstrates that the increase of the project duration can elevate the quality, but the marginal utility of the former is diminishing. Project cost, as exhibited in Figure 3, is nonlinearly related to the duration: a speedy working mode can dramatically increase the cost, but prolonging duration can hardly reduce the cost reversely. Figure 4 further explains the consequences of Figure 3: the growing cost leads to a lower quality, instead of a higher one, due to the shortened duration.

Figure 2: Relations between processing time and quality.

Figure 3: Relations between processing time and cost.

Figure 4: Relations between cost and quality.

Those solutions put processing time first without any preference suggestions and leave decision-makers struggling to choose the most appropriate ones. Therefore, we adopted data envelopment analysis (DEA) to further evaluate those solutions.

3. Results of DEA Models and Discussions

3.1. Results

In our DEA model, processing time is a reverse output. The smaller it is, the better. So for the sake of simplicity, we regarded the processing time, together with the cost, as the inputs, while the quality as the output. Given the inability of traditional DEA models, like CCR and BCC, to rank efficient units and thus to choose the best one, we utilized superefficiency DEA models to complete those works. Table 4 compares four efficiencies of the 32 options, including CCR, BCC, CRS superefficiency, and VRS superefficiency.

Table 4: Comparison of efficiencies between four DEA models.

Solutions
CCR
BCC
CRS
VRS

1
1.0000
1.0000
1.0000
1.0000

2
1.0000
1.0000
1.0000
1.0000

3
1.0000
1.0000
1.0000
1.0000

4
1.0000
1.0000
1.0000
1.0000

5
1.0000
1.0000
1.0000
1.0000

6
1.0000
1.0000
1.0000
1.0000

7
0.9976
1.0000
0.9976
1.0057

8
0.9985
1.0000
0.9985
1.0003

9
1.0000
1.0000
1.0000
1.0000

10
1.0000
1.0000
1.0000
1.0000

11
0.9845
1.0000
0.9845
1.0049

12
0.9961
1.0000
0.9961
1.0020

13
0.9974
1.0000
0.9974
1.0007

14
0.9950
0.9989
0.9950
0.9989

15
1.0000
1.0000
1.0000
1.0000

16
0.9418
1.0000
0.9418
1.0010

17
0.9530
1.0000
0.9530
1.0001

18
0.9707
1.0000
0.9707
1.0000

19
0.9935
1.0000
0.9935
1.0028

20
1.0000
1.0000
1.0000
1.0000

21
1.0000
1.0000
1.0003
1.0003

22
0.8972
0.9947
0.8972
0.9947

23
0.9190
0.9962
0.9190
0.9962

24
0.9260
0.9955
0.9260
0.9955

25
0.9546
0.9995
0.9546
0.9995

26
0.9816
0.9999
0.9816
0.9999

27
1.0000
1.0000
1.0002
1.0005

28
1.0000
1.0000
1.0015
inf

29
0.8972
0.9947
0.8972
0.9947

30
0.9016
0.9927
0.9016
0.9927

31
0.9151
0.9908
0.9151
0.9908

32
0.9267
0.9953
0.9267
0.9953

3.2. Discussions

Table 4 indicates that the CCR model results in 13 efficient units; the highest of the 32 efficiencies is 1 and the lowest is 0.8972. However, the BCC model obtains 22 efficient units; the highest is 1 and the lowest is 0.9908. These results imply that BCC holds a weaker ability than CCR of determining efficiency, although their minimum efficiencies are close. Among the results of the superefficiency CRS model, the highest is 1.0015 in decision-making unit (DMU) 28, and the lowest is similar to that of CCR, while for VRS model, the highest is 1.0057 in DMU 7, and the lowest equals that of BCC. CRS presents both lower maximum and lower minimum than VRS.

Notably, the maximums and the minimums of the four models are pretty close, which implies that finding the best from the 32 options is a tough task. But through comparing the four columns, one can notice that when a result of a traditional model is inefficient, it equals that of the corresponding superefficient model; otherwise, it is less than that of the corresponding superefficient model. This indicates that superefficient models can be used to further rank efficient units and find the best one. In addition, it should be noted that DMU 28 has no solution in the VRS superefficiency model, which is because we adopt an input-oriented super efficiency model, and the output value of DMU 28 (i.e., engineering quality value) is the largest among all DMU. Traditional models can help decision-makers find the worst inefficient DMU but tend to provide multiefficient DMUs with the same efficiency, due to the upper limit. Superefficiency models remove this limit and present efficient DMUs with various values; thus the best one can be easily spotted. In our study, CRS chose DMU 28 as the best while VRS chose DMU 7.

Superefficiency models can find the best among solutions to an MRCPSP in an effective and speedy way. Applying superefficiency DEA models to MRCPSP is validated to be a convincible and better approach.

MRCPSP occurs in many real-world applications, such as construction engineering, subway line selection, medical diagnosis, supplier location, cargo transportation, and software development. A very common difficulty of many real MRCPSPs is how to evaluate each of the solutions in a subjective way. The DEA approach proposed in this paper should be a good choice.

4. Conclusions

As project scheduling usually targets multigoals, solutions to multimode resource-constrained project scheduling problems can be multiple. How to select an appropriate one according to objective principles has become a problem in practice. We adopt data envelopment analysis to a typical MRCPSP and effectively propose a series of objective benchmarks that can help to evaluate those solutions to the MRCPSP and determine an appropriate one. Our approach provides an innovative perspective and an effective approach for decision-makers when selecting a proper solution to an MRCPSP.

Data Availability

The data used to support the findings of this study are included within the article.

Conflicts of Interest

The author has no conflicts of interest to declare.

Acknowledgments

This work was supported by the Xuzhou Medical University excellent talents scientific research start-up fund (53591420); initial funding for postdoctoral research of XZMU (53470321); and the Priority Academic Program Development of Jiangsu Higher Education Institutions (PAPD).

References

S. E. Elmaghraby, Activity Networks: Project Planning and Control by Network Models, Wiley, New York, NY, USA, 1977.

P. Brucker, A. Drexl, R. Möhring, K. Neumann, and E. Pesch, “Resource-constrained project scheduling: notation, classification, models, and methods,” European Journal Of Operational Research, vol. 112, no. 1, pp. 3–41, 1999.

R. Kolisch and A. Drexl, “Local search for nonpreemptive multi-mode resource-constrained project scheduling,” IIE Transactions, vol. 29, no. 11, pp. 987–999, 1997.

A. Schnell and R. F. Hartl, “On the efficient modeling and solution of the multi-mode resource-constrained project scheduling problem with generalized precedence relations,” Or Spectrum, vol. 38, no. 2, pp. 283–303, 2016.

M. H. Sebt, M. R. Afshar, and Y. Alipouri, “Hybridization of genetic algorithm and fully informed particle swarm for solving the multi-mode resource-constrained project scheduling problem,” Engineering Optimization, vol. 49, no. 3, pp. 513–530, 2017.

C. C. Liu, “A column generation based distributed scheduling algorithm for multi-mode resource constrained project scheduling problem,” Computers & Industrial Engineering, vol. 125, pp. 258–278, 2018.

R. Sonmez and M. Gurel, “Hybrid optimization method for large-scale multimode resource-constrained project scheduling problem,” Journal of Management in Engineering, vol. 32, no. 6, 2016.

R. Zamani, “An effective mirror-based genetic algorithm for scheduling multi-mode resource constrained projects,” Computers & Industrial Engineering, vol. 127, pp. 914–924, 2019.

F. Faghih-Mohammadi, A. Seifi, and M. Khalighi-Sikaroudi, “A modified modeling approach and a heuristic procedure for the multi-mode resource constrained project scheduling problem with activity splitting,” RAIRO-Operations Research, vol. 50, no. 1, pp. 91–118, 2016.

R. Nemati-Lafmejani, H. Davari-Ardakani, and H. Najafzad, “Multi-mode resource constrained project scheduling and contractor selection: mathematical formulation and metaheuristic algorithms,” Applied Soft Computing, vol. 81, p. 22, 2019.

A. Hill, E. Lalla-Ruiz, S. Voß, and M. Goycoolea, “A multi-mode resource-constrained project scheduling reformulation for the waterway ship scheduling problem,” Journal of Scheduling, vol. 22, no. 2, pp. 173–182, 2019.

B. O. Odedairo, “Relevance and applicability of multi-objective resource constrained project scheduling problem: review article,” Engineering, Technology & Applied Science Research, vol. 1, no. 6, pp. 144–150, 2011.

W. J. Gutjahr, “Bi-objective multi-mode project scheduling under risk aversion,” European Journal of Operational Research, vol. 246, no. 2, pp. 421–434, 2015.

E. B. Tirkolaee, A. Goli, M. Hematian, A. K. Sangaiah, and T. Han, “Multi-objective multi-mode resource constrained project scheduling problem using pareto-based algorithms,” Computing, vol. 101, no. 6, pp. 547–570, 2019.

H. Maghsoudlou, B. Afshar-Nadjafi, and S. T. A. Niaki, “A multi-objective invasive weeds optimization algorithm for solving multi-skill multi-mode resource constrained project scheduling problem,” Computers & Chemical Engineering, vol. 88, pp. 157–169, 2016.

M. Naldi, G. Nicosia, A. Pacifici, and U. Pferschy, “Profit-fairness trade-off in project selection,” Socio-Economic Planning Sciences, vol. 67, pp. 133–146, 2019.

P. Liu, B. Zhu, and P. Wang, “A multi-attribute decision-making approach based on spherical fuzzy sets for yunnan Baiyao's R&D project selection problem,” International Journal of Fuzzy Systems, vol. 21, no. 7, pp. 2168–2191, 2019.

A. Charnes, W. W. Cooper, and E. Rhodes, “Measuring the efficiency of decision making units,” European Journal of Operational Research, vol. 2, no. 6, pp. 429–444, 1978.

G. Debreu, “The coefficient of resource utilization,” Econometrica, vol. 19, no. 3, Journal of the Econometric Society, 1951.

T. C. Koopmans, “Analysis of production as an efficient combination of activities,” Activity Analysis of Production and Allocation, vol. 13, pp. 33–37, 1951.

M. J. Farrell, “The measurement of productive efficiency,” Journal of the Royal Statistical Society. Series A (General), vol. 120, no. 3, pp. 253–290, 1957.

R. Mahmoudi, A. Emrouznejad, S.-N. Shetab-Boushehri, and S. R. Hejazi, “The origins, development and future directions of data envelopment analysis approach in transportation systems,” Socio-Economic Planning Sciences, vol. 69, 2020.

S. Kohl, J. Schoenfelder, A. Fügener, and J. O. Brunner, “The use of data envelopment analysis (dea) in healthcare with a focus on hospitals,” Health Care Management Science, vol. 22, no. 2, pp. 245–286, 2019.

T. T. Yang, P. C. Wang, and F. Li, “Centralized resource allocation and target setting based on data envelopment analysis model,” Mathematical Problems in Engineering, vol. 10, 2018.

A. Mardani, “Data envelopment analysis in energy and environmental economics: an overview of the state-of-the-art and recent development trends,” Energies, vol. 11, no. 8, 2018.

H. Zhou, Y. Yang, Y. Chen, and J. Zhu, “Data envelopment analysis application in sustainability: the origins, development and future directions,” European Journal of Operational Research, vol. 264, no. 1, pp. 1–16, 2018.

S. Soheilirad, K. Govindan, A. Mardani, E. K. Zavadskas, M. Nilashi, and N. Zakuan, “Application of data envelopment analysis models in supply chain management: a systematic review and meta-analysis,” Annals of Operations Research, vol. 271, no. 2, pp. 915–969, 2018.

W. D. Cook and L. M. Seiford, “Data envelopment analysis (DEA) - thirty years on,” European Journal of Operational Research, vol. 192, no. 1, pp. 1–17, 2009.

J. S. Liu, L. Y. Y. Lu, W.-M. Lu, and B. J. Y. Lin, “A survey of dea applications,” Omega, vol. 41, no. 5, pp. 893–902, 2013.

M. Kapelko, “Measuring inefficiency for specific inputs using data envelopment analysis: evidence from construction industry in Spain and Portugal,” Central European Journal of Operations Research, vol. 26, no. 1, pp. 43–66, 2018.

E. Karasakal and P. Aker, “A multicriteria sorting approach based on data envelopment analysis for R&D project selection problem,” Omega, vol. 73, pp. 79–92, 2017.

M. Toloo, S. Nalchigar, and B. Sohrabi, “Selecting most efficient information system projects in presence of user subjective opinions: a dea approach,” Central European Journal of Operations Research, vol. 26, no. 4, pp. 1027–1051, 2018.

M. Toloo and M. Mirbolouki, “A new project selection method using data envelopment analysis,” Computers & Industrial Engineering, vol. 138, 2019.

Y. Wen, Q. An, X. Xu, and Y. Chen, “Selection of six sigma project with interval data: common weight dea model,” Kybernetes, vol. 47, no. 7, pp. 1307–1324, 2018.

V. Van Peteghem and M. Vanhoucke, “Using resource scarceness characteristics to solve the multi-mode resource-constrained project scheduling problem,” Journal of Heuristics, vol. 17, no. 6, pp. 705–728, 2011.

L. Wang and C. Fang, “An effective estimation of distribution algorithm for the multi-mode resource-constrained project scheduling problem,” Computers & Operations Research, vol. 39, no. 2, pp. 449–460, 2012.

F. Ballestín, A. Barrios, and V. Valls, “Looking for the best modes helps solving the mrcpsp/max,” International Journal of Production Research, vol. 51, no. 3, pp. 813–827, 2013.

S. X. Liu, D. Chen, and Y. F. Wang, “Memetic algorithm for multi-mode resource-constrained project scheduling problems,” Journal of Systems Engineering and Electronics, vol. 25, no. 4, pp. 609–617, 2014.