The stated choice (SC) experiment has been generally regarded as an effective method for behavior analysis. Among all the SC experimental design methods, the orthogonal design has been most widely used since it is easy to understand and construct. However, in recent years, a stream of research has put emphasis on the so-called efficient experimental designs rather than keeping the orthogonality of the experiment, as the former is capable of producing more efficient data in the sense that more reliable parameter estimates can be achieved with an equal or lower sample size. This paper provides two state-of-the-art methods called optimal orthogonal choice (OOC) and
Abundant and accurate data is the foundation of study. To date, emerging technologies are largely introduced in data mining and processing [
SC experiment has been widely used because it can observe choices on alternatives which do not exist in the current market. So analysts are able to predict, for example, the share rate of a newly introduced transportation mode. Another reason of its popularity lies in the ability to provide variability in attributes in a relatively small sample size compared with RP experiment, with which better estimation of influence of each attribute on choice can be achieved. Usually, respondents in a SC experiment will be faced with some “selected” choice situations, considering that making choices among all the possible combinations of attribute levels is too many to accomplish for a single respondent. Thus, how analysts distribute the levels of the design attributes in an experiment plays a big role. It may impact upon not only whether or not an independent assessment of contribution of each attribute to the choices observed can be determined, but also the ability of the experiment to detect statistical relationships that may exist within the data.
Historically, researchers have relied on orthogonal experimental designs, in which the attributes of the experiment are statistically independent by forcing them to be orthogonal [ Every level of every attribute appears at same times. All possible attribute level combinations of random two attributes appear at same times.
While orthogonal design has long been used in practice, in [
Acknowledgment of this fact has led researchers to transfer their efforts to obtain experimental designs that minimize the asymptotic variance-covariance (AVC) matrix of discrete choice models and provide more reliable parameter estimates with an equal or lower sample size. Such designs are called efficient designs. To date, most research has been focused on developing methods to generate efficient designs; comparison and validation of these methods in practice are rarely seen. This paper provides two state-of-the-art methods called optimal orthogonal choice (OOC) and
The remainder of this paper is organized as follows. In Section
Considering the popularity and convenience of orthogonal design for analyzers in practice, there is a stream of researchers in [
Generate a fractional factorial orthogonal design for alternative 1.
Choose some systematic changes to get the allocation of attribute levels in alternative 2 from alternative 1. Systematic changes are certain rules to decide how the attribute levels change from alternative 1 and will be discussed in later context.
Choose another systematic change to get the allocation of attribute levels in alternative 3 from alternative 1.
Keep doing this until all the alternatives are determined.
It will be much easier to understand this method by starting with a binary attribute level design. Again, we assume that
Optimal orthogonal choice design for 2 alternatives with 3 binary attributes.
Choice situation | Alternative 1 | Alternative 2 | ||||
---|---|---|---|---|---|---|
|
|
|
|
|
| |
1 | 0 | 0 | 0 | 1 | 1 | 1 |
2 | 0 | 1 | 1 | 1 | 0 | 0 |
3 | 1 | 0 | 1 | 0 | 1 | 0 |
4 | 1 | 1 | 0 | 0 | 0 | 1 |
To generate OOC design for more alternatives, it is necessary to introduce
In an example of OOC design for 3 alternatives and 3 attributes each with 2 levels shown in Table
We can see from Table
Optimal orthogonal choice design for 3 alternatives with 3 binary attributes.
Choice situation | Alternative 1 | Alternative 2 | Alternative 3 | ||||||
---|---|---|---|---|---|---|---|---|---|
|
|
|
|
|
|
|
|
| |
1 | 0 | 0 | 0 | 1 | 1 | 0 | 0 | 0 | 1 |
2 | 0 | 1 | 1 | 1 | 0 | 1 | 0 | 1 | 0 |
3 | 1 | 0 | 1 | 0 | 1 | 1 | 1 | 0 | 0 |
4 | 1 | 1 | 0 | 0 | 0 | 0 | 1 | 1 | 1 |
Designs for any choice set size with any number of attributes each having any number of levels can be generated in similar way. However, a big limitation of OOC design is that it can only generate designs for generic attributes according to the principle of this method. How alternative-specific attributes distribute across alternatives is rarely discussed in the literature. Another shortcoming of OOC design is that it may produce a lot of unreasonable combinations by forcing maximum level differences in attributes across alternatives. Answers from responders who have to make decision in such choice situations may not reflect their actual choice statements since the “proper alternative” may not be contained in the questionnaire.
While people who raise OOC method keep on improving design with remaining orthogonality, another stream of researchers goes straight forward to increasing the statistical efficiency of the design by minimizing the elements of the asymptotic variance-covariance (AVC) matrix of discrete choice models. The AVC matrix can be obtained by taking the negative inverse of the expected second derivatives of the log-likelihood function of the model proofed in [
Assume an individual faced with alternative
where
Considering that the most popular way to estimate parameters is maximum likelihood estimation, the log-likelihood function of parameters for a single respondent can be expressed as
Let
Since the calculation of
Figure
Comparison of investing in larger sample sizes versus more efficient designs.
Larger samples
Better designs
In this section, three experimental designs which used different methods (orthogonal, OOC, and
Prior parameter values and attribute levels for case study.
Prior parameter values | |||
---|---|---|---|
|
|
|
|
1 | −0.6 | −0.45 | −0.9 |
Attribute level | |||||
---|---|---|---|---|---|
Car | Taxi | Bus | |||
|
|
|
|
|
|
(min) | (RMB) | (min) | (RMB) | (min) | (RMB) |
15 | 4 | 15 | 16 | 25 | 1 |
20 | 5 | 20 | 18 | 30 | 2 |
30 | 7 | 30 | 22 | 38 | 3 |
In order to obtain better estimation of parameters, three levels are set for each attribute for maximum variation as much as possible. The values of
We generate three different (attribute level balanced) designs with 18 choice situations assuming the above MNL model, using the software Ngene 1.1.1. The design results are shown in Table
Experimental designs for empirical study.
Choice situation | Car | Taxi | Bus | Block | |||
---|---|---|---|---|---|---|---|
|
|
|
|
|
|
||
Orthogonal design for MNL model ( |
|||||||
1 | 15 | 4 | 15 | 16 | 25 | 1 | 1 |
2 | 20 | 5 | 20 | 18 | 30 | 2 | 1 |
3 | 30 | 7 | 30 | 22 | 38 | 3 | 1 |
4 | 15 | 5 | 20 | 22 | 38 | 1 | 1 |
5 | 20 | 7 | 30 | 16 | 25 | 2 | 1 |
6 | 30 | 4 | 15 | 18 | 30 | 3 | 1 |
7 | 30 | 7 | 20 | 18 | 25 | 1 | 2 |
8 | 15 | 4 | 30 | 22 | 30 | 2 | 2 |
9 | 20 | 5 | 15 | 16 | 38 | 3 | 2 |
10 | 20 | 7 | 15 | 22 | 30 | 1 | 2 |
11 | 30 | 4 | 20 | 16 | 38 | 2 | 2 |
12 | 15 | 5 | 30 | 18 | 25 | 3 | 2 |
13 | 30 | 5 | 30 | 16 | 30 | 1 | 3 |
14 | 15 | 7 | 15 | 18 | 38 | 2 | 3 |
15 | 20 | 4 | 20 | 22 | 25 | 3 | 3 |
16 | 20 | 4 | 30 | 18 | 38 | 1 | 3 |
17 | 30 | 5 | 15 | 22 | 25 | 2 | 3 |
18 | 15 | 7 | 20 | 16 | 30 | 3 | 3 |
|
|||||||
Orthogonal optimal design for MNL model ( |
|||||||
1 | 15 | 4 | 20 | 18 | 38 | 3 | 1 |
2 | 30 | 7 | 15 | 16 | 30 | 2 | 1 |
3 | 15 | 5 | 20 | 22 | 38 | 1 | 1 |
4 | 20 | 4 | 30 | 18 | 25 | 3 | 1 |
5 | 30 | 5 | 15 | 22 | 30 | 1 | 1 |
6 | 20 | 7 | 30 | 16 | 25 | 2 | 1 |
7 | 20 | 7 | 30 | 16 | 25 | 2 | 2 |
8 | 30 | 5 | 15 | 22 | 30 | 1 | 2 |
9 | 15 | 4 | 20 | 18 | 38 | 3 | 2 |
10 | 20 | 4 | 30 | 18 | 25 | 3 | 2 |
11 | 15 | 5 | 20 | 22 | 38 | 1 | 2 |
12 | 30 | 7 | 15 | 16 | 30 | 2 | 2 |
13 | 15 | 7 | 20 | 16 | 38 | 2 | 3 |
14 | 20 | 5 | 30 | 22 | 25 | 1 | 3 |
15 | 30 | 4 | 15 | 18 | 30 | 3 | 3 |
16 | 30 | 4 | 15 | 18 | 30 | 3 | 3 |
17 | 20 | 5 | 30 | 22 | 25 | 1 | 3 |
18 | 15 | 7 | 20 | 16 | 38 | 2 | 3 |
|
|||||||
|
|||||||
1 | 30 | 5 | 20 | 18 | 25 | 1 | 1 |
2 | 20 | 7 | 15 | 16 | 38 | 2 | 1 |
3 | 30 | 4 | 15 | 22 | 25 | 3 | 1 |
4 | 15 | 5 | 30 | 16 | 30 | 3 | 1 |
5 | 15 | 5 | 30 | 16 | 38 | 2 | 1 |
6 | 30 | 5 | 20 | 22 | 25 | 1 | 1 |
7 | 20 | 4 | 15 | 22 | 30 | 2 | 2 |
8 | 15 | 7 | 30 | 22 | 38 | 1 | 2 |
9 | 15 | 7 | 20 | 22 | 38 | 1 | 2 |
10 | 30 | 5 | 15 | 16 | 30 | 3 | 2 |
11 | 15 | 5 | 30 | 18 | 38 | 2 | 2 |
12 | 20 | 4 | 20 | 18 | 25 | 3 | 2 |
13 | 20 | 7 | 30 | 16 | 25 | 1 | 3 |
14 | 20 | 4 | 20 | 18 | 30 | 2 | 3 |
15 | 30 | 7 | 15 | 16 | 30 | 3 | 3 |
16 | 15 | 7 | 30 | 18 | 38 | 1 | 3 |
17 | 20 | 4 | 20 | 18 | 30 | 3 | 3 |
18 | 30 | 4 | 15 | 22 | 25 | 2 | 3 |
As expected, the two efficient designs produce lower
On the other hand, comparing the two efficient designs,
Further, most research focused on developing one of these experimental design methods by far. Though there may be a few of discussions about strength and shortcomings of every method separately, comparisons are rarely found in theory or practice area [
Comparison of orthogonal, OOC, and
Method | Advantage | Disadvantage |
---|---|---|
Orthogonal | (i) It is the most widely used method and easy to construct or obtain |
(i) There are too many choice situations/questions for a single respondent |
|
||
Optimal orthogonal choice | (i) Attribute level differences are maximized |
(i) It can only generate designs for generic attributes; the rules for setting up alternative-specific attributes are not clear right now |
|
||
|
(i) The smaller the asymptotic standard errors achieved, the smaller the width of the confidence intervals observed around the parameters estimates will be |
(i) In general not orthogonal (not that important) |
The SC experiment has been generally regarded as an effective method for discrete choice analysis, especially for newly introduced alternatives. The high cost on survey forces researchers to find more efficient design methods to obtain better estimation on parameters instead of investing a larger sample size. Though orthogonal design has been used as the major experimental design method, orthogonality is not that important in the nonlinear discrete choice models. In this paper, we provide two state-of-the-art efficient designs: OOC and
The authors declare that there is no conflict of interests regarding the publication of this paper.
This study is supported by the 2014 Doctoral Innovation Funds of Southwest Jiaotong University, the Fundamental Research Funds for the Central Universities (no. A0920502051307-03), Specialized Research Fund for the Doctoral Program of Higher Education (no. 20130184110020), National Natural Science Foundation of China (no. 51178403), and Sichuan Province Science and Technology Support Project (no. 2011FZ0050).