Due to the shortcomings in the traditional methods which dissatisfy the examination requirements in composing test sheet, a new method based on tabu search (TS) and biogeographybased optimization (BBO) is proposed. Firstly, according to the requirements of the testsheet composition such as the total score, test time, chapter score, knowledge point score, question type score, cognitive level score, difficulty degree, and discrimination degree, a multi constrained multiobjective model of testsheet composition is constructed. Secondly, analytic hierarchy process (AHP) is used to work out the weights of all the test objectives, and then the multiobjective model is turned into the single objective model by the linear weighted sum. Finally, an improved biogeographybased optimization—TS/BBO is proposed to solve testsheet composition problem. To prove the performance of TS/BBO, TS/BBO is compared with BBO and other populationbased optimization methods such as ACO, DE, ES, GA, PBIL, PSO, and SGA. The experiment illustrates that the proposed approach can effectively improve composition speed and success rate.
Examination is the process of teaching management which plays an important role in evaluating student achievement, inspiring students’ creativity, and improving student learning outcomes. As an important means of testing student learning and teacher teaching, it also provides the necessary information for teachers to improve their teaching methods and the quality of teaching [
Testsheet composition is a hot issue in computeraided testing. At present, the algorithms to solve testsheet composition are abductive machine learning [
Firstly proposed by Simon in 2008, biogeographybased optimization (BBO) [
The structure of this paper is organized as follows. Section
Testsheet composition model is a multiconstraint, multiobjective optimization problem. In this section, we will describe the mathematical model for testsheet composition, including constraints and objective function.
Automatic testsheet composition with computer is searching for questions to meet the requirements from item bank. We determine the encoding for the main properties according to objective analyzing and preestablish the goal state matrix
The process of composing testsheet is as follows. Firstly, find the row to meet the objectives and requirements of question combination
Every objective for testsheet composition corresponds to a constraint, all the
Total score
Test time
Chapter scores
Knowledge point score
Question type score
Cognitive level score
Difficulty degree
Discrimination degree
Through analysis, eight constraints of the testsheet composition model (total score, test time, chapter score, knowledge point score, question type score, cognitive level score, difficulty degree, and discrimination degree) should be equal to the evaluation requirements or with the minimum error of the evaluation requirements. Thus, the testsheet composition is a multiconstraint, multiobjective optimization problem. In practical application, we can consider the error between the above eight constraints and evaluation requirements as objective function
The mathematical model (i.e., objective function) for the testsheet composition is as follows:
In the above objective function
In practice, the weight
AHP [
Proposed by Zahedi [
AHP can be used to determine goal weight for the testsheet composition. Integrating the feature of testsheet composition into the AHP, specific solution process is shown in Algorithm
Level tree structure of testsheet composition.
We calculate consistency ratio
Comparison matrix







 


1  3  5  6  3  6  5  6 

1/3  1  2  5  1  3  2  3 

1/5  1/2  1  1  1/4  3  1  4 

1/6  1/5  1  1  1/6  1/3  1/5  1/3 

1/3  1  4  6  1  6  5  6 

1/6  1/3  1/3  3  1/6  1  1/5  1 

1/5  1/2  1  5  1/5  5  1  3 

1/6  1/3  1/4  3  1/6  1  1/3  1 
*
Weights
Weights 










0.7511  0.3098  0.1738  0.0700  0.4924  0.0914  0.2130  0.0930 

0.3423  0.1412  0.0792  0.0319  0.2244  0.0417  0.0970  0.0424 
Random index RI for different

3  4  5  6  7  8  9  10  11  12  13  14 

RI  0.58  0.90  1.12  1.24  1.32  1.41  1.45  1.49  1.51  1.54  1.56  1.57 
In the actual exam, testsheet goal weight is various. AHP can determine different testsheet goal weight responding to different requirements according to the different compassion matrix
Biogeography is the study of the migration, speciation, mutation, and extinction of species [
Firstly proposed by Simon in 2008, biogeographybased optimization (BBO) is a new evolution algorithm developed for the global optimization [
In BBO, migration operator can change existing habitat and modify existing solution. Migration is a probabilistic operator that adjusts habitat
Select
Select
Randomly select an SIV
Replace a random SIV in
Mutation is also a probabilistic operator that randomly modifies habitat SIVs based on the habitat a priori probability of existence. Very high HSI solutions and very low HSI solutions are equally improbable. Medium HSI solutions are relatively probable. The mutation rate
Additionally, the mutation operator tends to increase the population diversity. Mutation can be described in Algorithm
Compute the probability
Select SIV
Replace
and each habitat corresponding to a potential solution to the given problem.
Sort the population from best to worst.
For each habitat, map the HSI to the number of species
Calculate the immigration rate
Modify the population with the migration operator shown in Algorithm
Update the probability for each individual.
Mutate the population with the mutation operation shown in Algorithm
Evaluate the fitness for each individual in
Sort the population from best to worst.
Tabu search (TS) [
The candidate solution for the optimization problem under way is encoded by a configuration. Relying on the characteristic of the problem, the solution configuration
A move function
The move function defines a neighborhood which bounds the neighboring configurations which are reachable by implementing a move operation to the current solution configuration using
A tabu list which stops a recent move to be reversed is kept and modified in the TS framework. Once a move (say, swapping the
To allow the solution configuration to move to an attractive but tabu neighboring configuration, an aspiration criterion is strategically designed to overrule the tabu status of such a desired move. Aspiration criterion provides a restricted degree of freedom in accepting a tabu move that achieves a threshold of attractiveness.
The stopping criterion depends on the purpose of the problem. There are a number of alternatives such as a minimal solution quality level, a given CPU time limit, a maximal number of iterations between two improvements of the best solution found.
Basically, the TS approach can be summarized as shown in Algorithm
As we all know, the standard TS algorithm is good at exploring the search space and locating the region of global minimum, but it is relatively slow at exploitation of the solution. On the other hand, standard BBO algorithm is usually quick at the exploitation of the solution, though its exploration ability is relatively poor. Therefore, in our work, a hybrid metaheuristic algorithm by integrating tabu search into biogeographybased optimization, socalled TS/BBO, is used to solve the problem of testsheet composition. The difference between TS/BBO and BBO is that the hybrid migration operator is used to replace the original BBO mutation operator. In this way, this method can explore the new search space by the mutation of the TS algorithm and exploit the population information with the migration of BBO, and therefore it can overcome the lack of the exploitation of the TS algorithm. In the following, we will show the algorithm TS/BBO which is a variety of TS and BBO. Firstly, we describe tabu search migration and mutation operation, and then a mainframe of TS/BBO is shown.
The critical operator of TS/BBO is the tabu search migration operator, which composes the tabu search with the migration of BBO. In this algorithm, we can find that the migration between the population
Select
Select
ALLOW =
TS =
for
Randomly select an SIV
Replace a random SIV in
ALLOW = ALLOW
In the same way, we compose the tabu search and original mutation operator which modifies the habitat with low HSI in order to avoid the repeat to improve the search efficiency. Pseudocode of tabu search mutation operation can be described in Algorithm
Compute the probability
Select SIV
ALLOW =
TS =
Replace
ALLOW = ALLOW
By incorporating abovementioned tabu search migration and mutation operator into original BBO algorithm, the TS/BBO has been developed as a new algorithm. TS/BBO algorithm is given in Algorithm
In TS/BBO, the standard continuous encoding of TS/BBO cannot be used to solve testsheet composition directly. In order to apply TS/BBO to testsheet composition, one of the key issues is to construct a direct relationship between the testsheet sequences and the vector of individuals in TS/BBO.
We will do the following preprocessing before the design of TS/BBO algorithm for testsheet composition
We regroup all the questions in item bank according to question type, and then the candidates take the same question type together to form a subset. So all the questions in item bank can be divided into several different subsets, and then all questions will be renumbered.
Because question type in the requirements and item bank has the same score, we can calculate the number of questions required for every question type, and then we get the total number of questions in a testsheet.
When using TS/BBO to solve testsheet composition problem, the status code for each habitat represents a candidate solution, that is, a testsheet composition scheme. Therefore, how to determine the effective habitat status code is a key issue.
Traditional encoding method is as follows: the status code of each habitat is represented by a binary string whose length is the number of questions in total in item bank, and the number “1” indicates that the question corresponding to the number is selected, while number “0” indicates that the question corresponding to the number is not selected; the length of the number “1” indicates the number of questions contained in total in a testsheet. This encoding method is simple and maximizes a random search at most, but it has increased the search space, reducing the search efficiency.
Therefore, an alternative encoding method is proposed in this paper. This method rearranges questions into different subsets according to the different question type. The status code of each habitat is represented by an
For example, we have seven questions
Improved BBO can adapt to the needs of testsheet, while optimization algorithms can improve the BBO fast search capabilities and increase the search to the global possible optimum solution. HSI in Habitat
Based on the above analysis, the pseudo code of improved BBOTS/BBO for testsheet composition is described as shown in Algorithm
In this section, we look at the performance of TS/BBO as compared with BBO and other populationbased optimization methods. Firstly, we compare performances between BBO and TS/BBO, and then we compare performances between TS/BBO and other populationbased optimization methods such as ACO, DE, ES, GA, PBIL, PSO, and SGA.
To allow a fair comparison of running times, all the experiments were performed on a PC with a Pentium IV processor running at 3.0 GHz, 1 GB of RAM and a hard drive of 160 Gbytes. Our implementation was compiled using MATLAB R2012a (7.14) running under Windows XP. In the following, we will describe the problem we use to test the performance of the TS/BBO.
We randomly generate an item bank with
the total score is 100;
the test time is 120 minutes;
chapter score {1 : 2 : 3 : 4 : 5 : 6 : 7 : 8 : 9 : 10
knowledge point score {1 : 2 : 3 : 4 : 5 : 6 : 7 : 8 : 9 : 10:11:12:13:14:15:16:17:18:19:20} = {3 : 3 : 3 : 3 : 4 : 4 : 6 : 6 : 4 : 5 : 6 : 6 : 6 : 6 : 6 : 9 : 5 : 5 : 5 : 5} (a chapter contains two knowledge points);
question type score {Type 1 : Type 2 : Type 3 : Type 4} = {10 : 16 : 24: 50};
cognitive level score
difficulty degree is 0.4900;
discrimination degree is 0.4050.
We get the goal weights for testsheet composition {total score : test time : chapter score : knowledge point score : questions type score : cognitive level score : difficulty degree : discrimination degree
For BBO and TS/BBO, we used the same following parameters: habitat modification probability = 1, immigration probability bounds per gene =
Best normalized optimization results on testsheet composition problem. The numbers shown are the best results found after 100 Monte Carlo simulations of BBO and TS/BBO algorithm.
Parameter  Algorithm  




BBO  TS/BBO 
50  50  10  3.2243 

50  50  2  3.8055 

100  50  2 

3.8957 
100  100  2  3.4048 

100  100  10  3.5727 

Worst normalized optimization results on testsheet composition problem. The numbers shown are the best results found after 100 Monte Carlo simulations of BBO and TS/BBO algorithm.
Parameter  Algorithm  




BBO  TS/BBO 
50  50  10 

5.3541 
50  50  2 

5.6907 
100  50  2 

5.5479 
100  100  2 

4.8595 
100  100  10 

4.5499 
Mean normalized optimization results on testsheet composition problem. The numbers shown are the minimum objective function values found by BBO and TS/BBO algorithm, averaged over 100 Monte Carlo simulations.
Parameter  Algorithm  




BBO  TS/BBO 
50  50  10  4.5200 

50  50  2  4.9518 

100  50  2  4.7500 

100  100  2  4.2129 

100  100  10  4.4600 

Average CPU time on testsheet composition problem. The numbers shown are the minimum average CPU time (Sec) consumed by BBO and TS/BBO algorithm.
Parameter  Algorithm  




BBO  TS/BBO 
50  50  10  5.80 

50  50  2  4.24 

100  50  2  9.19 

100  100  2 

17.78 
100  100  10 

26.01 
From Table
In order to explore the benefits of TS/BBO, in this section we compared its performance on testsheet composition problem with seven other populationbased optimization methods. ACO (ant colony optimization) [
We did some fine tuning on each of the optimization algorithms to get optimal performance, to get the optima for every algorithm. For ACO, we used the following parameters: initial pheromone value
To compare the different effects among the parameters
Best normalized optimization results on testsheet composition problem. The numbers shown are the best results found after 100 Monte Carlo simulations of each algorithm.
Parameter  Algorithm  




ACO  DE  ES  GA  PBIL  PSO  SGA  TS/BBO 
50  50  10  5.4134  5.1592 
5.5487  4.1308  6.7791  5.3212  3.5948 

50  50  2  6.3357  5.8272 
6.1941  5.3657  8.1954  6.1651  4.4489 

100  50  2  5.9669 
5.6499 
5.9860  5.1919  7.9242  5.9516  3.9447 

100  100  2  5.6849 
5.2775 
5.6952  4.6971  7.8740  5.6846  3.4552 

100  100  10  5.7449 
5.2911 
5.7152  4.0889  7.6467  5.6881  3.5391 

Worst normalized optimization results on testsheet composition problem. The numbers shown are the best results found after 100 Monte Carlo simulations of each algorithm.
Parameter  Algorithm  




ACO  DE  ES  GA  PBIL  PSO  SGA  TS/BBO 
50  50  10  7.1331  6.7607 
7.2079  6.2192 

7.0046  6.0518 

50  50  2  7.0307  6.5340 
6.9859  6.6226 

6.8968 

5.6907 
100  50  2  6.8203 
6.4536 
6.9457  6.3000 

7.0567 

5.5479 
100  100  2  6.6193 
6.1777 
6.5664  5.9887 

6.5046 

4.8595 
100  100  10  6.7001 
6.1475 
6.5471  5.0184 

6.5030  4.7275 

Mean normalized optimization results on testsheet composition problem. The numbers shown are the minimum objective function values found by each algorithm, averaged over 100 Monte Carlo simulations.
Parameter  Algorithm  




ACO  DE  ES  GA  PBIL  PSO  SGA  TS/BBO 
50  50  10  6.4966  6.0329 
6.4322  5.1371  8.7371  6.4414  4.7929 

50  50  2  6.5258  6.0020 
6.3800  5.5267  8.4413  6.3501 

4.7521 
100  50  2  6.3249 
5.6499 
6.3452  5.5034  8.3996  6.3087  4.1814 

100  100  2  6.0259 
5.2775 
6.0369  4.9790  8.3465  6.0256 

4.0310 
100  100  10  6.0896 
5.2911 
6.0581  4.3342  8.1055  6.0294  3.7515 

Average CPU time on testsheet composition problem. The numbers shown are the minimum average CPU time (Sec) consumed by each algorithm.
Parameter  Algorithm  




ACO  DE  ES  GA  PBIL  PSO  SGA  TS/BBO 
50  50  10  6.37  6.37 
4.52  3.87  3.55  4.27  4.94 

50  50  2  6.36  6.36 
3.56  3.90  4.56  4.17  4.38 

100  50  2  13.51 
26.09 
7.83  8.54  7.96  10.47  10.36 

100  100  2  26.09 
26.09 
15.64  17.35  15.40  20.52  22.69 

100  100  10  26.03 
26.03 
15.51  16.79  18.34  20.41  23.61 

From Table
The simulation implemented in this section shows that the algorithm TS/BBO that we proposed performed the best and most effectively, and it can solve the testsheet problem perfectly.
To improve performance for testsheet composition, we combined the advantage of tabu search and biogeographybased optimization and proposed a new algorithm TS/BBO. Simulation experiment demonstrates that AHP and TS/BBO that we proposed for testsheet composition optimization problem have the following advantages.
Speed up extracting questions and compute the objective function after preprocessing the item bank according to question type.
Habitat encoding length is equal to the total number of testsheet, saving storage space and reducing the optimization space, to ensure that the constraint for total score is met and improve the solution accuracy.
We use subencoding according to the question type to ensure that constraint for question type score is satisfied, reducing the optimization space and improving the solution accuracy.
AHP determines the testsheet composition weights, comprehensively considering the objective and subjective factors, in line with the actual test environment, fitting real test needs.
Tabu search optimizes mutation and migration operator to create the algorithm TS/BBO, improving solution efficiency and solution accuracy.
However, the algorithm TS/BBO that we proposed in this paper has the following disadvantages: the preprocessing time increases when the number of questions in item bank is getting bigger, which will affect the entire testsheet composition speed; need further optimization to improve accuracy; need for further ease the conflict between expanding population diversity and reducing the optimization space. The above problems are worth further study.