Mathematical Problems in Engineering
Volume 2017 (2017), Article ID 2819795, 16 pages
https://doi.org/10.1155/2017/2819795

Research Article

Implications of a Spatial Multicriteria Decision Analysis for Urban Development in Ulaanbaatar, Mongolia

Purevtseren Myagmartseren,¹ Myagmarsuren Buyandelger,² and S. Anders Brandt³

¹Department of Geography, School of Arts and Science, National University of Mongolia, 14201-46A Ulaanbaatar, Mongolia
²Land Administration Department of Capital City, 15160 Ulaanbaatar, Mongolia
³Department of Industrial Development, IT and Land Management, Faculty of Engineering and Sustainable Development, University of Gävle, 801 76 Gävle, Sweden

Correspondence should be addressed to S. Anders Brandt; sab@hig.se

Received 28 October 2016; Accepted 26 December 2016; Published 15 February 2017

Academic Editor: Huade Guan

Copyright © 2017 Purevtseren Myagmartseren et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract.

New technology has provided new tools for effective spatial planning. Through the example of locating suitable sites for urban development in Ulaanbaatar, this paper illustrates how multicriteria decision analysis and geographical information systems can be used for more effective urban planning. Several constraint and factor criteria were identified, transformed into map layers, and weighted together using the analytic hierarchy process. Besides localization results, this study shows the effect of using poor elevation data and how a sensitivity analysis can be applied to yield further information, spot weighting weaknesses, and assess the quality of the criteria.

1. Introduction

1.1. Background

Spatial planning for urban development is always associated with different types of decision-making problems and strategies, and the outcomes of planning will clearly differ depending on who the stakeholders and analysts are, together with which approaches and techniques are used. Until as recent as the last decade, most spatial planning activities worldwide have been done without the use of geographic information systems (GIS) and multicriteria decision analysis (MCDA) technologies, and even today they are rarely combined, despite the fact that most planning organizations have sufficient computer and software resources. For example, Bhatta [1, p. 57] stated “that compared to the swift adoption of GIS, urban analysts have been slower to embrace technical developments in spatial analysis.” The reason for this lack of use of spatial-analytical tools that may be beneficial for planning can partly be linked to a shortage of domain-specific expertise among spatial planners, or as Berke et al. [2, p. 91] put it: “Without a planning staff skilled in computer applications, the new technology is of little use.” Most critical is “the ‘soft’ side of the ‘technology package’: technique, organization, and knowledge” (Klosterman [3, p. 4]). Therefore, nowadays many university programs in planning try to incorporate GIS in their program curricula, but so far most programs do not show any explicit content of the combination of GIS and MCDA in their course syllabi. Although there is a slow increase of using these tools by university students and researchers, to a large extent planning practitioners are still unaware of the opportunities these tools provide, especially in developing countries.

1.2. Multicriteria Decision Analysis for Spatial Planning

One of the biggest challenges local governments and community groups face today is to find new areas for development in a sustainable way, without degrading land values. GIS and MCDA together provide tools for handling such natural resource management issues (National Research Council [4]; Malczewski [5]; Chen et al. [6]). In short, GIS-based MCDA takes data (input) and combines it into a weighted map (output). As GIS is capable of handling and combining different types of data (both nonspatial and spatial as well as multitemporal and multiscale) in a time-efficient and cost-effective way, there has been a steady increase in interest for using GIS together with MCDA techniques (Chen et al. [6]). Furthermore, the previous conventional map overlay approaches have been considerably advanced thanks to the integration of MCDA techniques with GIS (Malczewski [5]), and therefore local governments and community groups, among others, have started to implement the technology together with more sophisticated spatial analysis (National Research Council [4]). For successful implementation of spatial MCDA two things are of critical importance (Malczewski [5, p. 33]):

(i)“The GIS capabilities of data acquisition, storage, retrieval, manipulation, and analysis”(ii)“The MCDM capabilities for combining the geographical data and the decision maker’s preferences into unidimensional values of alternative decisions”

With the advent of powerful personal computers, practical MCDA tools, and user-friendly GIS, a number of studies have been carried out showing the usefulness of these techniques. With respect to land management, already in the 1960s the Land Inventory branch of the Canadian government and the Harvard Lab for Computer Graphics and Spatial Analysis created a GIS for developing land management plans (Malczewski [5]). Examples from the 80s and 90s include Lyle and Stutz [7], who looked at urban suitability modeling, and Miller et al. [8], who looked at potential sites for greenway development in a town, and more recent ones encompass Berke et al. [2], Carr and Zwick [9], Jain and Subbaiah [10], Liu et al. [11], Zucca et al. [12], Yang et al. [13], Önüt et al. [14], Park et al. [15], and Rikalovic et al. [16].

1.3. Aims

Mongolia is a post-communist country, and together with many others, like post-Soviet states, planning practitioners have been used to plan the cities without any involvement of citizens, making the planning process both very straightforward, fast, and without any interruptions in the initial project periods. Today, many of these countries have new legislation requiring public participation input and at the same time the technology and higher education may have fallen behind. As to our knowledge, no scientific studies have been reported on physical planning in Mongolia where spatial MCDA is used. Together with the fact that most spatial MCDA studies are carried out by specialists, not working in regular planning departments in the public sector, and for many developing countries they do not even exist, this justifies the need for examples illustrating the advantages and opportunities these tools provide. Then the planning process can benefit from technology (technical point of view), as well as from understanding of the underlying problem and democratic perspectives (social points of view).

Taking Ulaanbaatar as an example, the general scope of this paper is to bring practitioners, especially those in developing countries, knowledge of the opportunities of combining MCDA with GIS. This will be achieved through fulfilling the following objectives:

(i)Illustrate how MCDA and GIS can be used in the planning process for new urban areas, in order to raise the awareness among planners when strong domain knowledge is lacking, and present a strategy for locating new urban areas.(ii)Identify relevant criteria for carrying out a spatial MCDA for new urban area development.(iii)Compare different resolution of the digital elevation data to see how results are affected.(iv)Illustrate how the sensitivity analysis can contribute to see if the relative weights of criteria are robust.

The structure of the paper is as follows: Section 2 contains the methods descriptions; Section 3 describes the results; Section 4 finishes with discussion and conclusions.

2. Methods

2.1. Study Area

To achieve the aims of the paper, Ulaanbaatar, the capital city of Mongolia, and its surroundings are used as an example. The city is located in northeastern Mongolia, in the narrow valley of Tuul River in the Khentii Mountain Range (Figure 1), at the boundary of taiga, mountain forest, and steppe.

Figure 1: Map over the study area. (a) Location of Ulaanbaatar; (b) TIN model with rivers and urban areas (gray = built-up area; light green–brown = TIN).

With annual mean temperature of −1.3°C and minimum temperatures often below −30°C, Ulaanbaatar is one of the coldest cities of Mongolia as well as the coldest capital city in the world. Although summers are warm (July average 16.9°C), winters are cold (January average −22.3°C). Total annual precipitation is on average 281.7 mm, which mostly comes as summer rain (>82% in June–September) and as small amounts of snow in the cold season (Sato [17]).

With respect to land use, the city has undergone a dramatic urbanization trend during the last decade (the urban land use distribution of Ulaanbaatar city is presented in Table 1). Using high-resolution satellite image fusion interpretation, Amarsaikhan et al. [18] concluded that the central part of Ulaanbaatar city was urbanized very rapidly. In the socialist time, 1930–1990, 571 buildings were constructed, while during the first years of post-socialist time, 1990–2008, 792 buildings were raised, which demonstrates a significant land demand for urban development. Herders and farmers, many of them migrating from rural places, have played a big role in this urban expansion (sprawl). Just half a decade ago, in 1956, the city’s population was 118,000, but today 1.3 million residents, out of Mongolia’s 3 million in total, are living in the Ulaanbaatar region. One key factor for such rapid urbanization is probably due to the nomadic life style of Mongolian people; that is, it is not more difficult to put up a traditional ger tent in the outskirts of a city than it is to put it up anywhere else.

Table 1: Urban land use in Ulaanbaatar for 2001 and 2012 (data from Purevtseren et al. [19] and Myagmartseren et al. [20]).

Land use
% 2001
% 2012
Percentage points change

Residential (total)
33.2
58.7
25.5

Residential (sprawl)
28.2
51.8
whereof 32% ger districts
and 19.8% in green belt
23.6

Infrastructure
5.2
9.4
4.2

Industry
10.0
8.3
−1.7

Service
13.7
6.3
−7.4

Special purpose and others (defense, public land, water buffer, mining, etc.)
6.9
15.1
8.2

Unused and reservation
28.8
10.0
−18.8

In the residential zone ger area, which covers about 32% percent of all territory of the city, urban expansion has accelerated so far so it adversely impacts green-belt areas, wetlands, water buffer zones, open space, and public land (see how the red area in Figure 2 is changing between 2001 and 2010 and also that it is mainly spreading in the narrow valleys out from the city center).

Figure 2: Urban areas (red) in the Ulaanbaatar city area, Mongolia, in 2000 and 2010. Light green color is forest, green is wetland, mandarin is arable land, and gray is pasture and others (adapted from source: cadastral map and time series land use planning data in National Land Information Database).

2.2. Workflow and Criteria Treated

This study consisted of two main stages: data processing and MCDA (Figure 3). In the first stage, relevant criteria were identified with respect to physical considerations and the minimum requirements found in Ulaanbaatar urban plan and vision documents, and in the second stage the actual analysis was performed.

Figure 3: Overview of workflow for this study.

For the locational and physical criteria, example tables for standard residential urban development are provided by, for example, McDonald et al. [21], Rowe et al. [22], Wells and King [23], USDA [24], FAO [25], and NSW Environment Protection Authority [26]. These sources were used as inspiration for this study (Table 2). Based on Table 2 and other criteria, the constraints shown in Table 3 were considered.

Table 2: Suitability criteria for urban development.

Land attribute
Suitability criteria of factor

Unsuitable
Moderate with limitations
Most suitable

Slope gradient (%)
>15
8–15
<8

Soil texture
Sand, silt
Loamy sand, sandy clay loam, silt clay loam
Loam, clay loam, silt loam

Soil character
Moving sand, undeveloped soil, boggy, saline, rock
Water, wind eroded, content of stones, low humus, permafrost, and so forth
No evidence of erosion, low amounts of stone, sufficient humus content, and so forth

Distance from road (m)
<100; >3000
1500–3000
100–1500

Aspect (degrees)
North 0–22.5; 337.5–360
West 22.5–112.5
East 247.5–337.5
South 112.5–247.5

Close to built-up area (m)
>3000
1500–3000
150–1500

Close to river (m)
<500; >3000
1500–3000
500–1500

Vegetation cover (NDVI index)
<0.2
0.2–0.3
0.3–0.8

Department of Housing [27].

City and village planning and construction norms and rules (no. 30-01-04) of Mongolia [28].

Table 3: Urban development location factor and constraints criteria.

Criteria: urban development site
Consideration

Must not be located in or near agricultural areas
Safety

Must avoid forest reserved areas
Environment/resources

Must be on a gentle slope (i.e., <15%)
Safety/environment

Must avoid wetlands
Safety/environment

Must be near to rivers, but not in buffer zone
Economic/environment

Must have a sufficient amount of green grasses (NDVI)
Economic

Must not be located in national parks
Environment/resources

Must have optimal amount of solar radiation (aspect)
Economic/environment

Must have an optimal soil for construction and garden
Environment/resources

2.3. Analytic Hierarchy Process

Unless all criteria are of Boolean data type, that is, either true or false, some kind of weighting needs to be employed to be able to combine them. Malczewski [29] noted that in the last 20 years of scientific works related to the integration of MCDA with GIS there have been a number of multicriteria decision rules invented. However, some are used more frequent. Out of the total 259 surveyed papers treating multiattribute decision analysis, 143 were of the type Boolean overlay or weighted summation. A similar, but more developed method is the analytic hierarchy process (AHP) developed by Saaty [30], which uses additive weighting and also can derive the weights associated with the map layers as well as being able to aggregate and include the level representing alternatives into the hierarchy structure (Malczewski [29]). Furthermore, AHP is capable of relating both tangible and intangible criteria.

To derive values for criteria weights, we treated all factor maps as if they belonged to the same hierarchy level (using the freeware software AHP 2.0 by Brandt [31]). In AHP, as some criteria may be relatively more important than other criteria, the criteria are compared pairwise against each other in order to get relative weights for each of them. Then the final suitability score of each pixel in the final map can be calculated from

where is the suitability with respect to the objective being considered, is the relative weight of factor , and is the criteria score of factor . Through this weighted linear combination of factors, the sum of all weights will be 1, and all weights can be directly associated with their corresponding percent of importance. The criteria score, which also contains similar elements of importance, is determined after a normalization procedure (cf. Section 2.4.2).

In this study, all pairwise comparison weights were arbitrarily decided by us. The slope factor was assigned the highest weight. Thereafter, distance to already existing urban areas, soil type, distance to roads, distance to rivers, vegetation cover type, and finally the terrain aspect were assigned progressively lower weights (Table 4).

Table 4: Ranking and weighting of factors (resulting consistency ratio is 0.021).

Rank
Criterion
Slope
Urban
Soil
Road
River
Veg.
Aspect
Relative weight

1
Slope gradient
1
1
2
3
4
4
10
0.307

2
Urban distance
1
1
1
2
3
4
4
0.221

3
Soil
1/2
1
1
1
2
4
4
0.173

4
Road distance
1/3
1/2
1
1
1
2
3
0.116

5
River distance
1/4
1/3
1/2
1
1
1
2
0.081

6
Vegetation cover
1/4
1/4
1/4
1/2
1
1
1
0.058

7
Aspect
1/10
1/4
1/4
1/3
1/2
1
1
0.044

2.4. GIS Processing

2.4.1. Preparation of GIS Layers

To be able to produce both the factor maps (listed in Table 4) and constraint maps, which indicate if the areas can be considered at all, a number of data sources and methods to create the information layers were used (Table 5). Most information was collected from a subset of a Landsat 7 satellite image of September 2010 (free download available from http://landsat.usgs.gov/landsat-data-access) and radar-based SRTM elevation data of 2001 from the CGIAR Consortium (free download available from http://srtm.csi.cgiar.org/) and optical stereo-based ASTER GDEM elevation data (free download available from http://asterweb.jpl.nasa.gov/gdem.asp), but also other sources were used for data and information input.

Table 5: Data sources used to create the various information layers for the criteria maps.

Features considered
Source
Method of creation

Urban areas
Landsat 7 satellite image
Unsupervised classification

Forest areas
Landsat 7 satellite image
Supervised classification

Topography (slope, aspect)
CGIAR-CSI DEM of 90 m resolution/Aster GDEM of 30 m
Import via GIS (grid) and Erdas (raster)

Hydrology
ASTER GDEM
Geoprocessing tool/hydrology tool in GIS

Vegetation
Landsat 7 satellite image
Normalized difference vegetation index (NDVI)

National parks
National parks map
Digitizing/georeferencing

2.4.2. Production of Constraint and Factor Maps

In spatial MCDA studies usually two different types of maps, or information layers, are used: constraint and factor maps. Constraint maps are of the Boolean data type, meaning that they consist of cells with only two possible values, zeroes (0) or ones (), or in other words, false or true. When such maps are multiplied with each other, for a particular cell to result with the value 1, all layers have to contain that number. If as much as one layer contains the value 0, also the resulting map will have the value 0 for that cell, meaning that it is not possible at all to utilize for the intended purpose (in this case impossible to use for urban development). In this study constraint maps were produced for barren lands, forests, rivers including surrounding buffer zone, wetlands, national parks, and slopes that are too steep.

Factor maps, on the other hand, show for each factor each cell’s suitability with respect to the goal (in this case urban development). Initially the factor maps can contain values of any type or size. For example, distance to agricultural areas may be represented by cell values in meters and soil types by plain text. To be able to analyze such different types of data together, all data need to be converted to numerical format as well as being normalized; that is, the value of high suitability with respect to one factor (criterion) has to have the same value as one of another factor. Therefore, to represent the suitability, all factors were normalized, meaning in this case that the value ranges of all factor maps were stretched from 0 to 255. This means that 0 is considered to be of very low suitability and 255 of very high suitability, irrespectively of factor type (criteria). The factor maps produced were slope gradient, urban distance, soil, road distance, river distance, vegetation cover, and aspect. As they all contained the whole range of values between 0 and 255, the maps produced could serve as continuous representations of distances to objects, suitability, and so forth.

2.4.3. Executing the MCDA

To execute the MCDA, all constraint maps were multiplied with each other, producing one combined constraint map. The factor maps, on the other hand, first had to be multiplied with their corresponding weights from the AHP and, thereafter, summed together into one combined factor map. Finally, the combined constraint map and the combined factor map were multiplied together. The general equation is

where is the total suitability score, is the weight corresponding to , that is, factor map , and is constraint map . Now every pixel in the final result map contains a suitability score for urban development.

2.5. Sensitivity Analysis

As many scientists argue that real-world decisions are uncertain because we have insufficient information or fuzzy descriptions of the semantic meaning of the events, phenomena, statements themselves, and so forth, the results of MCDA are usually associated with uncertainties or direct errors or undesirable consequences (Aerts et al. [32]; Malczewski [29]; Tenerelli and Carver [33]). Among errors and variability in model choice, system understanding, weighting factors, data used, and human judgment, one of the greatest contributors to controversy and uncertainty is the criteria weights, which also may have a significant impact on the results (Feizizadeh and Blaschke [34]).

One way to account for such uncertainties is to perform a sensitivity analysis that aims to assess the response of a model to changes in input parameters (Crosetto et al. [35]; Gómez Delgado and Bosque Sendra [36]; Ligmann-Zielinska and Jankowski [37]; Chen et al. [6]). Saltelli et al.’s [38, p. 45] definition of sensitivity analysis is “the study of how the uncertainty in the output of a model (numerical or otherwise) can be apportioned to different sources of uncertainty in the model input.” Then it will be possible to see “‘which of the uncertain input factors is more important in determining the uncertainty in the output of interest?’ or ‘if we could eliminate the uncertainty in one of the input factors, which factor should we choose to reduce the most the variance of the output?’” (Saltelli et al. [38, p. ix]).

There are different types of sensitivity methods that can be used, for example, probabilistic Monte Carlo simulations (e.g., Aerts et al. [32]; Benke and Pelizaro [39]), the Variance-based Global Sensitivity Analysis (GSA) method (Homma and Saltelli [40]), or representing weights with different degrees of fuzziness (Kordi and Brandt [41]). However, among the most popular sensitivity analysis methods is the One-Factor-At-A-Time (OAT) (cf. Daniel [42, 43]). This method changes the value of only one factor at a time, while keeping all other factor values constant. Although the method suffers from some limitations, there are a number of advantages compared with other methods: () it is intuitive (i.e., the analyst can see if a change by some percentage of one factor will result in a corresponding output change), () the results from the sensitivity analysis can be instantaneously visualized and explored, () no prior knowledge about OAT is required, and () it is computationally effective and does not require huge amounts of model executions (Ligmann-Zielinska and Jankowski [44]). Usually one of the following is done to analyze the criteria sensitivity: changing criteria values, changing relative importance of criteria, or changing criteria weights (Chen et al. [6]).

To test the robustness of our MCDA model and to see how the analyzed urban development locations change when small deviations in the criteria values are applied, a sensitivity analysis was performed by changing each of the factor criteria weight values at a time (OAT), that is, changing the values by ±6, ±12, and ±18%, respectively, from the original value (Table 6). Correspondingly, all other criteria lost proportional shares according to the equation given by Chen et al. [6]:

where is the weight of the th criterion at a certain percent level, is the weight of the main changing criterion at a certain percent level, is the weight of the th criterion for the base map run, and is the weight of the main changing criterion , for the base map run.

Table 6: Criteria weights in sensitivity analysis when one factor’s weight is changed from original weight (i.e., base map) by −18, −12, −6, 6, 12, or 18%.

Change input
Weight values

Slope
Urban
Soil
Road
River
Vegetation
Aspect

Base map
0.3072
0.2215
0.1731
0.1157
0.0807
0.0582
0.0436

Slope −18%
0.2519
0.2392
0.1869
0.1249
0.0871
0.0628
0.0471

Slope −12%
0.2703
0.2333
0.1823
0.1219
0.0850
0.0613
0.0459

Slope −6%
0.2888
0.2274
0.1777
0.1188
0.0828
0.0597
0.0448

Slope +6%
0.3256
0.2156
0.1685
0.1126
0.0786
0.0567
0.0424

Slope +12%
0.3441
0.2097
0.1639
0.1095
0.0764
0.0551
0.0413

Slope +18%
0.3625
0.2038
0.1593
0.1065
0.0743
0.0536
0.0401

Urban −18%
0.3229
0.1816
0.1820
0.1216
0.0848
0.0612
0.0458

Urban −12%
0.3177
0.1949
0.1790
0.1197
0.0835
0.0602
0.0451

Urban −6%
0.3124
0.2082
0.1761
0.1177
0.0821
0.0592
0.0443

Urban +6%
0.3020
0.2348
0.1701
0.1137
0.0793
0.0572
0.0429

Urban +12%
0.2967
0.2481
0.1672
0.1117
0.0779
0.0562
0.0421

Urban +18%
0.2915
0.2614
0.1642
0.1098
0.0766
0.0552
0.0414

Soil −18%
0.3188
0.2298
0.1419
0.1201
0.0837
0.0604
0.0452

Soil −12%
0.3149
0.2271
0.1523
0.1186
0.0827
0.0597
0.0447

Soil −6%
0.3111
0.2243
0.1627
0.1172
0.0817
0.0589
0.0441

Soil +6%
0.3033
0.2187
0.1835
0.1142
0.0797
0.0575
0.0431

Soil +12%
0.2995
0.2159
0.1939
0.1128
0.0787
0.0567
0.0425

Soil +18%
0.2956
0.2132
0.2043
0.1113
0.0777
0.0560
0.0420

Road −18%
0.3144
0.2267
0.1772
0.0949
0.0826
0.0596
0.0446

Road −12%
0.3120
0.2250
0.1758
0.1018
0.0820
0.0591
0.0443

Road −6%
0.3096
0.2232
0.1745
0.1088
0.0813
0.0587
0.0439

Road +6%
0.3048
0.2198
0.1717
0.1226
0.0801
0.0577
0.0433

Road +12%
0.3024
0.2180
0.1704
0.1296
0.0794
0.0573
0.0429

Road +18%
0.3000
0.2163
0.1690
0.1365
0.0788
0.0568
0.0426

River −18%
0.3121
0.2250
0.1758
0.1175
0.0662
0.0591
0.0443

River −18%
0.3104
0.2238
0.1749
0.1169
0.0710
0.0588
0.0441

River −6%
0.3088
0.2227
0.1740
0.1163
0.0759
0.0585
0.0438

River +6%
0.3056
0.2203
0.1722
0.1151
0.0855
0.0579
0.0434

River +12%
0.3040
0.2192
0.1713
0.1145
0.0904
0.0576
0.0431

River +18%
0.3023
0.2180
0.1704
0.1139
0.0952
0.0573
0.0429

Vegetation −18%
0.3106
0.2240
0.1750
0.1170
0.0816
0.0477
0.0441

Vegetation −12%
0.3095
0.2231
0.1744
0.1166
0.0813
0.0512
0.0439

Vegetation −6%
0.3083
0.2223
0.1737
0.1161
0.0810
0.0547
0.0438

Vegetation +6%
0.3061
0.2207
0.1725
0.1153
0.0804
0.0617
0.0434

Vegetation +12%
0.3049
0.2199
0.1718
0.1148
0.0801
0.0652
0.0433

Vegetation +18%
0.3038
0.2190
0.1712
0.1144
0.0798
0.0687
0.0431

Aspect −18%
0.3097
0.2233
0.1745
0.1166
0.0814
0.0587
0.0358

Aspect −12%
0.3089
0.2227
0.1740
0.1163
0.0811
0.0585
0.0384

Aspect −6%
0.3080
0.2221
0.1736
0.1160
0.0809
0.0584
0.0410

Aspect +6%
0.3064
0.2209
0.1726
0.1154
0.0805
0.0580
0.0462

Aspect +16%
0.3055
0.2203
0.1722
0.1151
0.0803
0.0579
0.0488

Aspect +18%
0.3047
0.2197
0.1717
0.1148
0.0800
0.0577
0.0514

3. Results

3.1. Result of MCDA

Combined constraint maps were produced (containing the product of barren lands, forests, rivers including surrounding buffer zones, wetlands, national parks, and slopes that are too steep) for two different cases of DEM resolution (Figure 4). If as much as one criterion’s value is 0 for a particular pixel, that pixel’s resulting value will also be 0 and considered not possible for urban development (black areas in Figure 4). Furthermore, the effect of resolution can also be seen; for example, small constraint areas may appear within the suitable areas for the 30 m DEM, while the same areas for the 90 m DEM show up as larger contiguous suitable sites.

(a)

(b)

Figure 4: Final constraint maps where the slope criterion comes from (a) high-resolution DEM of 30 m and (b) low-resolution DEM of 90 m.

Also combined factor maps were produced (containing the weighted sum of slope gradient, urban distance, soil, road distance, river distance, vegetation cover, and aspect) for the two different DEM resolutions (Figure 5). In these maps, all pixels have values that represent the total suitability, excluding the possibility of constraint occurrence. The main difference between them can be seen in the valleys where the low-resolution DEM leads to larger areas of high-ranked suitability.

(a)

(b)

Figure 5: Final factor maps where the elevation related criteria come from (a) high-resolution DEM of 30 m and (b) low-resolution DEM of 90 m.

Finally, for both sets of DEM resolution, the combined constraint map was multiplied with the combined factor map to produce final suitability maps for urban development (Figure 6).

(a)

(b)

Figure 6: Final suitability maps where the elevation related criteria come from (a) high-resolution DEM of 30 m and (b) low-resolution DEM of 90 m.

3.2. Sensitivity Analysis

The results from the sensitivity analysis are presented in Table 7, which shows how many cells there are in each suitability class ( = low suitability, = medium suitability, and = high suitability), as well as how many and percentage of the cells that have shifted class compared with the base map.

Table 7: For different levels of change of one criterion (cf. Table 6), the number of cells in each suitability class, and the number and percentage of cells that have shifted class compared with the base map.

Change input
Sensitivity map
⁢Difference between sensitivity map and base map

1
2
3
⁢1
⁢2
⁢3

(0–84)
(>84–168)
(>168–255)
⁢(0–84)
⁢ (>84–168)
⁢ (>168–255)

(# cells)
(# cells)
(# cells)
(# cells)
(%)
(# cells)
(%)
(# cells)
(%)

Base
644239
1493369
214143
0
0
0
0
0
0

Slope −18%
557807
1477549
316395
−86432
−13.4
−15820
−1.1
102252
47.7

Slope −12%
584823
1482800
284128
−59416
−9.2
−10569
−0.7
69985
32.7

Slope −6%
614137
1491049
246565
−30102
−4.7
−2320
−0.2
32422
15.1

Slope +6%
676054
1489162
186535
31815
4.9
−4207
−0.3
−27608
−12.9

Slope +12%
711534
1476143
164074
67295
10.4
−17226
−1.2
−50069
−23.4

Slope +18%
725447
1471445
154859
81208
12.6
−21924
−1.5
−59284
−27.7

Urban −18%
748595
1429919
173237
104356
16.2
−63450
−4.2
−40906
−19.1

Urban −12%
711589
1454280
185882
67350
10.5
−39089
−2.6
−28261
−13.2

Urban −6%
677243
1475205
199303
33004
5.1
−18164
−1.2
−14840
−6.9

Urban +6%
612625
1509268
229858
−31614
−4.9
15899
1.1
15715
7.3

Urban +12%
582593
1522757
246401
−61646
−9.6
29388
2.0
32258
15.1

Urban +18%
554993
1531849
264909
−89246
−13.9
38480
2.6
50766
23.7

Soil −18%
626473
1530169
195109
−17766
−2.8
36800
2.5
−19034
−8.9

Soil −12%
631098
1519343
201310
−13141
−2.0
25974
1.7
−12833
−6.0

Soil −6%
637251
1507001
207499
−6988
−1.1
13632
0.9
−6644
−3.1

Soil +6%
651642
1479202
220907
7403
1.1
−14167
−0.9
6764
3.2

Soil +12%
659334
1463997
228420
15095
2.3
−29372
−2.0
14277
6.7

Soil +18%
666484
1448725
236542
22245
3.5
−44644
−3.0
22399
10.5

Road −18%
614767
1540564
196420
−29472
−4.6
47195
3.2
−17723
−8.3

Road −12%
624185
1525776
201790
−20054
−3.1
32407
2.2
−12353
−5.8

Road −6%
634313
1509594
207844
−9926
−1.5
16225
1.1
−6299
−2.9

Road +6%
654300
1476911
220540
10061
1.6
−16458
−1.1
6397
3.0

Road +12%
664963
1459479
227309
20724
3.2
−33890
−2.3
13166
6.1

Road +18%
675633
1441832
234286
31394
4.9
−51537
−3.5
20143
9.4

River −18%
675076
1474173
202502
30837
4.8
−19196
−1.3
−11641
−5.4

River −12%
665153
1480465
206133
20914
3.2
−12904
−0.9
−8010
−3.7

River −6%
654294
1487117
210340
10055
1.6
−6252
−0.4
−3803
−1.8

River +6%
634151
1499720
217880
−10088
−1.6
6351
0.4
3737
1.7

River +12%
623218
1506307
222226
−21021
−3.3
12938
0.9
8083
3.8

River +18%
613213
1512423
226115
−31026
−4.8
19054
1.3
11972
5.6

Vegetation −18%
639094
1479907
232750
−5145
−0.8
−13462
−0.9
18607
8.7

Vegetation −12%
634103
1487175
230473
−10136
−1.6
−6194
−0.4
16330
7.6

Vegetation −6%
649638
1486011
216102
5399
0.8
−7358
−0.5
1959
0.9

Vegetation +6%
638737
1500874
212140
−5502
−0.9
7505
0.5
−2003
−0.9

Vegetation +12%
633322
1508804
209625
−10917
−1.7
15435
1.0
−4518
−2.1

Vegetation +18%
628014
1516474
207263
−16225
−2.5
23105
1.5
−6880
−3.2

Aspect −18%
634247
1491502
226002
−9992
−1.6
−1867
−0.1
11859
5.5

Aspect −12%
638020
1492129
221602
−6219
−1.0
−1240
−0.1
7459
3.5

Aspect −6%
640834
1492891
218026
−3405
−0.5
−478
0.0
3883
1.8

Aspect +6%
647603
1493941
210207
3364
0.5
572
0.0
−3936
−1.8

Aspect +12%
650659
1494456
206636
6420
1.0
1087
0.1
−7507
−3.5

Aspect +18%
654526
1494613
202612
10287
1.6
1244
0.1
−11531
−5.4

To illustrate how the variation of criteria weights impacts the analysis, the sensitivity maps were compared with the original base map (Figure 7). The most suitable locations are shown in white and light gray and the unsuitable locations in dark gray to black. Considering the S3 class, most changes occurred for the two cases when slope gradient weight was decreased by 18% and urban distance weight was increased with 18%, that is, resulting in 47.7% and 23.7% areal change, respectively. Spatially most changes occurred in the north part of the Ulaanbaatar city vicinity. Particularly the valleys show pixels changing to or from the class.

Figure 7: Result of sensitivity analysis over the northern part of the study area when one criterion is changed either +18 or −18%. White areas represent pixels that belonged to class from the beginning and pixels that have changed to or from class . The cell change percentage value refers to the percent pixels that changed to or from class.

In the sensitivity analysis, although many pixels changed class to and from class , out of the 14 pixels in the base map with very high suitability scores (where value ≥ 250) nine pixels still retained such high scores after a weight-change scenario of ±18% for all factors (Figure 8). This implies that the MCDA in fact is more robust than the statistics in Table 7 express.

(a)

(b)

Figure 8: Location of pixels with the highest suitability scores (value ≥ 250) in sensitivity analysis. (a) The nine pixels that are present in all sensitivity scenarios. (b) Remaining high score pixels appearing at a variety of locations for the different sensitivity scenarios.

3.3. Data Validation: Comparison with City Master Plan

Good reference or ground truth data are required for validation of the results of the MCDA. Provided that the earlier developed urban development zone map of Ulaanbaatar (i.e., the land management department of Ulaanbaatar’s city master plan with legal binding urban development zones (Myagmartseren et al. [45])) was based on sound planning strategies and reliable data, an accuracy assessment could be carried out where the results of this MCDA were compared to that urban development zone map. The urban development zone map has three zones: (i) a zone for development of high density, (ii) a zone for development of low density, and (iii) a zone prohibited for development. The third zone consists of preurban agriculture, green-belt forests, or other natural conservation uses, as well as areas where soil, land, water, distance to other features, and so forth are very unsuitable for urban development and if utilized would require vast resources in terms of additional planning and special engineering measures. Therefore, it is both likely and desirable that these areas remain for conservation purposes only.

To keep comparisons simple, the suitability maps were reclassified into two classes: one suitable class () with values 84–256 and one unsuitable class () with values 0–84. For comparison, the suitable class was matched to the zone for development and the unsuitable class to nondevelopment zones according to the urban development zone map of Ulaanbaatar (Figure 9). Then a confusion/error matrix (Table 8) was calculated, including the commonly used overall accuracy, producer’s accuracy, user’s accuracy measurement, and Kappa analysis (Congalton [46]).

Table 8: Error matrix for the accuracy assessment.

Suitability range values
Reference cells
Suitability compared cells
Matched cells
Producer’s accuracy%
Users accuracy%

Suitable: (84–255)
932193
1035752
879891
85
94

Unsuitable: (0–84)
1413414
843566
791264
94
56

Total cells
2345607

1671155

Overall accuracy: 0.71.

Kappa coefficient: 0.46.

(a)

(b)

Figure 9: Comparison of the (a) final suitability map reclassified into two classes with the (b) urban development zone map of Ulaanbaatar, serving as reference in the accuracy assessment.

As the overall accuracy is 0.71, meaning that 71% of the pixels are identically classified in both maps, it appears that the final suitability map resembles previous land use planning represented by the urban development zone map. The Kappa coefficient, on the other hand, is relatively low. A value of 0.46 indicates that the classified map is about 46% better than a random assignment of these classes. However, it should be noted that the previous development zone map, besides pure suitability concerns, also is based on subjective thoughts and visions of planners, considerations of local people (e.g., that preurban local communities may have refused development activity), and others.

By extracting spatially contiguous pixels with the highest suitability values, especially areas located in the western parts of Ulaanbaatar were found particularly interesting for future development (Figure 10).

(a)

(b)

Figure 10: Identified suitable sites for urban development where (a) the suitability score is 84–255 and (b) where contiguous pixels of that score have an area larger than 0.05 km².

4. Discussion and Conclusions

In Mongolia, during the last decade, the use of remotely sensed data and simple overlaying techniques has been routinely practiced for urban land suitability evaluation and new settlement site selections. However, together with spatial MCDA techniques, environmental and land management activities may be more precise, quick, and cost effective. Furthermore, MCDA has also been shown to have a significant demonstrative effect for decision makers; hence the general scope of this paper is to bring practitioners knowledge of the opportunities of combining MCDA with GIS. As the results of this study clearly demonstrate the advantages of considering the opportunities using MCDA with GIS for urban development, these techniques should be considered mandatory in larger projects. For example, urban planners may be demanded to use detailed and classified suitability maps, such as those proposed by Hannam and Hicks [47], or to describe all constraints and factors in detail in the legends of maps (as van Gool et al. [48] did for land resources mapping). Building on the above, the following discussion will proceed from the specific objectives outlined in Section 1.3.

By following the general structure of this paper, a straightforward approach is presented on how to utilize MCDA and GIS in the planning process for locating new urban areas (Objective 1). The procedure generally starts with requests from politicians or official authority personnel to develop a city in one or another way. After the goals of development have been determined, it is the analyst’s job to gather information on which criteria are needed for the analysis and get access to or prepare production of necessary data. The next step usually requires external input to be able to rank or weight the criteria against each other. Depending on the character of the criteria, this may be a straightforward and relatively easy task, or it may be a delicate one requiring well-thought analyses, maybe involving both experts and the general public, that may require special techniques to be able to combine diverse opinions (cf., e.g., Forman and Peniwati [49]; Matsatsinis and Samaras [50]; Jaganathan et al. [51]; Chakhar and Saad [52]; Shi et al. [53]). After the weights have been determined, the analyst produces the different map layers that represent the weighted criteria and performs the MCDA GIS analysis. Consecutively, an uncertainty analysis can be made if reference data are available to estimate inaccuracies, and through varying the weights within relevant ranges a sensitivity analysis can be carried out. Finally, the resulting suitability map has to be scrutinized for possible flaws and the most suitable locations have to be checked through field visits to see if the model successfully represents the reality or if there are other issues that may hinder further progress (e.g., cadastral or land-ownership issues).

Regarding identifying relevant criteria (Objective 2), the criteria listed in Section 2.2 seem to be generally accepted in previous research literature for this kind of study, as well as being feasible to carry out. However, it is also obvious that this study could have benefited from other or additional criteria. For example, geotechnical data were not used despite their importance regarding ground stability issues in the urban site selection process, nor were local people consulted for participation in the evaluation. Using local people’s community knowledge about their land and their needs is a key issue in land use planning. In this respect, also planning agencies should be considered as they may be a conduit to resources external to a community. Including these kinds of criteria would most likely have resulted in changed weights in the AHP. Still, the reason for not including these criteria in this study was mainly due to shortage of data and the time factor, as the general purpose was not to produce the perfect urban development plan, but rather to provide an approach for using MCDA and GIS in planning. Furthermore, in general the use of remotely sensed data was shown to be very effective for production of many criteria layers, but also the methods used to create the GIS layers could have an effect on the criteria used in the analysis. For example, unsupervised classification, at least in this particular study, did provide better results for urban areas. But it did not clearly identify forest and open spaces, where, on the other hand, supervised classification performed better. It may also be wise to consider what will happen if the area is larger, necessitating even more criteria, if the area will provide countless alternatives for development, or if the best locations are even more suitable for other types of activities and should be kept for those.

To derive the slope and aspect GIS layers, two different elevation datasets of 30 m and 90 m spatial resolution were used, enabling a comparison to see how results were affected (Objective 3). These datasets are available for free and for developing countries thus provide popular alternatives to more costly data collection methods, provided the resolution is adequate. As most criteria in this study were derived from 30 m Landsat ETM imagery, better resolution than 30 m seemed to be of no value. However, if the study area had been smaller and the analysis had required higher resolution, these elevation datasets, as well as satellite images, may have been too poor. When comparing the results of using the 90 m DEM with the sensitivity analysis, where only the 30 m DEM was used, it can be seen that the lower resolution DEM affected the final result in a greater way than did the change in relative criteria weights. Besides the general changes, a coarser DEM resolution led to larger contiguous areas of high-ranked suitability. This can be directly linked to the cell size’s influence on the slope. Big cell sizes may smoothen local terrain variation, thereby resulting in lower slopes than small cell sizes would produce. However, in mountainous terrain, local pockets of flat terrain may produce higher scores for small cell sizes. Therefore, it is recommended to try to find DEMs with as high resolution as possible, while keeping in mind that the other criteria have to be of about the same resolution to take advantage of the DEM.

Through the sensitivity analysis, it could be determined if the model itself and the relative weights of the criteria were robust (Objective 4). In general, although sensitive to weight value changes, all criteria weights as well as the entire model can be considered to be robust. As expected, the largest number of pixels moving to or from the highest suitability class occurred when the criterion of the largest relative weight, that is, slope, was tested. Roughly, the percent change of pixels belonging to S3 then was twice the size of the change in weight. With smaller relative weights, also the number of pixels changing suitability classes was smaller (e.g., vegetation cover and aspect). Geographically, it can be seen that most changes occurred in the northern part of the Ulaanbaatar city vicinity. This may be due to that two criteria—slope and urban distances—were largely constrained by the narrow valleys and localization of previous infrastructure, thereby abruptly changing the suitability of a pixel within short distance ranges. Therefore, it is of importance to first look at which criteria are capable of producing significantly changed results when exposed for weight changes in a sensitivity analysis. Secondly, it is important to try to minimize the uncertainties related to the weight determination of those criteria. However, as new areas of the highest suitability class may show up, the sensitivity maps could also function as additional tools for finding potential expansion areas for urban development.

When the result of the analysis was compared with the existing development plans, the general pattern of suitable locations coincides. However, the Kappa coefficient reveals that there is a disagreement between them. The main difference is that previous plans have dedicated a much larger area for future development. The reason could be due to different ways of thinking between a GIS expert and planners, with respect to territory development. The planners, for example, may have thought more on policies, local community responses, and visions rather than the actual conditions, which form the basis in the GIS analysis. If the difference is too large between these kinds of analyses and the vision of planners, there may be a risk of low implementation or even abandonment of using these kinds of analyses. Therefore, it is desirable to include both the planners and their visions into the MCDA analyses and the opposite—including MCDA in the planning process.

Competing Interests

The authors declare that there are no competing interests regarding the publication of this paper.

Acknowledgments

Thanks are due to all people that have helped the authors during this study, and special thanks go to the Land Administration Department of Capital City and Lanres Company who allowed them to use their digitized data. S. A. Brandt has also been financially supported by the EU through Tillväxtverket (Project GLOBES 2, no. 170430).

References

B. Bhatta, Analysis of Urban Growth and Sprawl from Remote Sensing Data, Advances in Geographic Information Science, Springer, Heidelberg, Germany, 2010.

P. R. Berke, D. R. Godschalk, E. J. Kaiser, and D. A. Rodriguez, Urban Land Use Planning, University of Illinois Press, Urbana, Ill, USA, 5th edition, 2006.

R. Klosterman, “Planning support systems: a new perspective,” in Planning Support Systems: Integrating Geographic Information Systems, Models, and Visualization Tools, R. Brail and R. Klosterman, Eds., pp. 263–284, ESRI Press, Redlands, Calif, USA, 2001.

National Research Council, GIS for Housing and Urban Development, National Academies Press, Washington, DC, USA, 2003.

J. Malczewski, “GIS-based land-use suitability analysis: a critical overview,” Progress in Planning, vol. 62, no. 1, pp. 3–65, 2004.

Y. Chen, J. Yu, and S. Khan, “Spatial sensitivity analysis of multi-criteria weights in GIS-based land suitability evaluation,” Environmental Modelling & Software, vol. 25, no. 12, pp. 1582–1591, 2010.

J. Lyle and F. P. Stutz, “Computerised land use suitability mapping,” Cartographic Journal, vol. 20, no. 1, pp. 39–49, 1983.

W. Miller, M. G. Collins, F. R. Steiner, and E. Cook, “An approach for greenway suitability analysis,” Landscape and Urban Planning, vol. 42, no. 2–4, pp. 91–105, 1998.

M. H. Carr and P. D. Zwick, Smart Land-Use Analysis: The LUCIS Model: Land Use Conflict Identification Strategy, ESRI Press, Redlands, Calif, USA, 2007.

K. Jain and Y. V. Subbaiah, “Site suitability analysis for urban development using GIS,” Journal of Applied Sciences, vol. 7, no. 18, pp. 2576–2583, 2007.

Y. Liu, X. Lv, X. Qin et al., “An integrated GIS-based analysis system for land-use management of lake areas in urban fringe,” Landscape and Urban Planning, vol. 82, no. 4, pp. 233–246, 2007.

A. Zucca, A. M. Sharifi, and A. G. Fabbri, “Application of spatial multi-criteria analysis to site selection for a local park: a case study in the Bergamo Province, Italy,” Journal of Environmental Management, vol. 88, no. 4, pp. 752–769, 2008.

F. Yang, G. Zeng, C. Du, L. Tang, J. Zhou, and Z. Li, “Spatial analyzing system for urban land-use management based on GIS and multi-criteria assessment modeling,” Progress in Natural Science, vol. 18, no. 10, pp. 1279–1284, 2008.

S. Önüt, T. Efendigil, and S. S. Kara, “A combined fuzzy MCDM approach for selecting shopping center site: an example from Istanbul, Turkey,” Expert Systems with Applications, vol. 37, no. 3, pp. 1973–1980, 2010.

S. Park, S. Jeon, S. Kim, and C. Choi, “Prediction and comparison of urban growth by land suitability index mapping using GIS and RS in South Korea,” Landscape and Urban Planning, vol. 99, no. 2, pp. 104–114, 2011.

A. Rikalovic, I. Cosic, and D. Lazarevic, “GIS based multi-criteria analysis for industrial site selection,” Procedia Engineering, vol. 69, pp. 1054–1063, 2014.

H. Sato, “Mongolia: the water situation in Ulaanbaatar,” Social System Review, vol. 3, pp. 55–63, 2012.

D. Amarsaikhan, V. Battsengel, E. Egshiglen, R. Gantuya, and D. Enkhjargal, “Applications of GIS and very high-resolution RS data for urban land use change studies in mongolia,” International Journal of Navigation and Observation, vol. 2011, Article ID 314507, 8 pages, 2011.

G. Purevtseren, P. Myagmartseren, and S. Jigjidsuren, Master Land Use Plan of Ulaanbaatar City, Governor Office of Capital City, Ulaanbaatar, Mongolia, 2001.

P. Myagmartseren, I. Myagmarjav, and B. Erdenejargal, “Land use changes in urban areas of mongolia,” in Proceedings of the 9th International Conference Environment and Sustainable Development in Mongolian Plateau and Surrounding Regions, A. K. Tulokhonov, E. Zh. Garmaev, and A. S. Mikheeva, Eds., vol. 2, pp. 18–21, Buryat State University, Ulan Ude, Russian Federation, August 2013.

R. C. McDonald, R. F. Isbell, J. G. Speight, J. Walker, and M. S. Hopkins, Australian Soil and Land Survey Field Handbook, Inkata Press, Melbourne, Australia, 2nd edition, 1990.

R. K. Rowe, D. F. Howe, and N. F. Alley, Guidelines for Land Capability Assessment in Victoria, Land Capability Assessment Section, Soil Conservation Authority, Victoria, 1981.

M. R. Wells and P. D. King, “Land capability assessment methodology for rural-residential development and associated agricultural land uses,” Land Resources Series no. 1, Western Australian Department of Agriculture, Perth, Australia, 1989.

USDA, National Soils Handbook. National Cooperative Soil Survey, 430-VI-NSH, United States Department of Agriculture, Government Printer, Washington, DC, USA, 1983.

FAO, Guidelines: Land Evaluation for Rainfed Agriculture, FAO Soils Bulletin No 52, FAO, Rome, Italy, 1983.

NSW Environment Protection Authority, Managing Urban Stormwater: Council Handbook. Draft, Environment Protection Authority, Sydney South, 1997.

Department of Housing, Techniques for Soil & Water Management at Building Sites, Department of Housing, Sydney, Australia, 1992.

Ministry of Infrastructure of Mongolia, City and Village Planning and Construction Norms and Rules (No 30-01-04) of Mongolia, Ministry of Infrastructure of Mongolia, Ulaanbaatar, Mongolia, 2004.

J. Malczewski, “GIS-based multicriteria decision analysis: a survey of the literature,” International Journal of Geographical Information Science, vol. 20, no. 7, pp. 703–726, 2006.

T. L. Saaty, “A scaling method for priorities in hierarchical structures,” Journal of Mathematical Psychology, vol. 15, no. 3, pp. 234–281, 1977.

S. A. Brandt, “AHP v. 2.0,” Analytic hierarchy process software, 2006, http://sab.geovega.se/lattjo.html.

J. C. J. H. Aerts, M. F. Goodchild, and G. B. M. Heuvelink, “Accounting for spatial uncertainty in optimization with spatial decision support systems,” Transactions in GIS, vol. 7, no. 2, pp. 211–230, 2003.

P. Tenerelli and S. Carver, “Multi-criteria, multi-objective and uncertainty analysis for agro-energy spatial modelling,” Applied Geography, vol. 32, no. 2, pp. 724–736, 2012.

B. Feizizadeh and T. Blaschke, “An uncertainty and sensitivity analysis approach for GIS-based multicriteria landslide susceptibility mapping,” International Journal of Geographical Information Science, vol. 28, no. 3, pp. 610–638, 2014.

M. Crosetto, S. Tarantola, and A. Saltelli, “Sensitivity and uncertainty analysis in spatial modelling based on GIS,” Agriculture, Ecosystems & Environment, vol. 81, no. 1, pp. 71–79, 2000.

M. Gómez Delgado and J. Bosque Sendra, “Sensitivity analysis in multicriteria spatial decision-making: a review,” Human and Ecological Risk Assessment, vol. 10, no. 6, pp. 1173–1187, 2004.

A. Ligmann-Zielinska and P. Jankowski, “A framework for sensitivity analysis in spatial multiple criteria evaluation,” in Geographic Information Science: 5th International Conference, GIScience 2008, Park City, UT, USA, September 23–26, 2008. Proceedings, T. J. Cova, H. J. Miller, K. Beard, A. U. Frank, and M. F. Goodchild, Eds., vol. 5266 of Lecture Notes in Computer Science, pp. 217–233, Springer, Berlin, Germany, 2008.

A. Saltelli, S. Tarantola, F. Campolongo, and M. Ratto, Sensitivity Analysis in Practice, John Wiley & Sons, Chichester, UK, 2004.

K. K. Benke and C. Pelizaro, “A spatial-statistical approach to the visualisation of uncertainty in land suitability analysis,” Journal of Spatial Science, vol. 55, no. 2, pp. 257–272, 2010.

T. Homma and A. Saltelli, “Importance measures in global sensitivity analysis of nonlinear models,” Reliability Engineering and System Safety, vol. 52, no. 1, pp. 1–17, 1996.

M. Kordi and S. A. Brandt, “Effects of increasing fuzziness on analytic hierarchy process for spatial multicriteria decision analysis,” Computers, Environment and Urban Systems, vol. 36, no. 1, pp. 43–53, 2012.

C. Daniel, “On varying one factor at a time,” Biometrics, vol. 14, no. 3, pp. 430–431, 1958.

C. Daniel, “One-at-a-time plans,” Journal of the American Statistical Association, vol. 68, no. 342, pp. 353–360, 1973.

A. Ligmann-Zielinska and P. Jankowski, “Spatially-explicit integrated uncertainty and sensitivity analysis of criteria weights in multicriteria land suitability evaluation,” Environmental Modelling & Software, vol. 57, pp. 235–247, 2014.

P. Myagmartseren, I. Myagmarjav, and G. Gantulga, Amendment of Master Land Use Plan of Ulaanbaatar City, Land Administration Department of Capital City, Ulaanbaatar, Mongolia, 2012.

R. G. Congalton, “Putting the map back in map accuracy assessment,” in Remote Sensing and GIS Accuracy Assessment, R. S. Lunetta and J. G. Lyon, Eds., pp. 1–13, CRC Press, Boca Raton, Fla, USA, 2004.

I. D. Hannam and R. W. Hicks, “Soil conservation and urban land use planning,” Journal, Soil Conservation Service of New South Wales, vol. 36, no. 3, pp. 134–145, 1980.

D. van Gool, P. Tille, and G. Moore, Land Evaluation Standards for Land Resource Mapping, Resource Management Technical Report 298, Department of Agriculture, Government of Western Australia, 3rd edition, 2005.

E. Forman and K. Peniwati, “Aggregating individual judgments and priorities with the Analytic Hierarchy Process,” European Journal of Operational Research, vol. 108, no. 1, pp. 165–169, 1998.

N. F. Matsatsinis and A. P. Samaras, “MCDA and preference disaggregation in Group Decision Support Systems,” European Journal of Operational Research, vol. 130, no. 2, pp. 414–429, 2001.

S. Jaganathan, J. J. Erinjeri, and J.-I. Ker, “Fuzzy analytic hierarchy process based group decision support system to select and evaluate new manufacturing technologies,” The International Journal of Advanced Manufacturing Technology, vol. 32, no. 11-12, pp. 1253–1262, 2007.

S. Chakhar and I. Saad, “Incorporating stakeholders’ knowledge in group decision-making,” Journal of Decision Systems, vol. 23, no. 1, pp. 113–126, 2014.

S. Shi, J. Cao, L. Feng, W. Liang, and L. Zhang, “Construction of a technique plan repository and evaluation system based on AHP group decision-making for emergency treatment and disposal in chemical pollution accidents,” Journal of Hazardous Materials, vol. 276, pp. 200–206, 2014.