Advances in Meteorology
Volume 2015 (2015), Article ID 472523, 22 pages
http://dx.doi.org/10.1155/2015/472523

Research Article

Using Artificial Intelligence to Retrieve the Optimal Parameters and Structures of Adaptive Network-Based Fuzzy Inference System for Typhoon Precipitation Forecast Modeling

Chien-Lin Huang,¹ Nien-Sheng Hsu,¹ Chih-Chiang Wei,² and Chun-Wen Lo¹

¹Department of Civil Engineering, National Taiwan University, No. 1, Section 4, Roosevelt Road, Taipei 10617, Taiwan
²Department of Digital Content Designs and Management, Toko University, No. 51, Section 2, University Road, Pu-Tzu City, Chiayi County 61363, Taiwan

Received 4 July 2014; Accepted 1 October 2014

Academic Editor: Sahra Kacimi

Copyright © 2015 Chien-Lin Huang et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract.

This study aims to construct a typhoon precipitation forecast model providing forecasts one to six hours in advance using optimal model parameters and structures retrieved from a combination of the adaptive network-based fuzzy inference system (ANFIS) and artificial intelligence. To enhance the accuracy of the precipitation forecast, two structures were then used to establish the precipitation forecast model for a specific lead-time: a single-model structure and a dual-model hybrid structure where the forecast models of higher and lower precipitation were integrated. In order to rapidly, automatically, and accurately retrieve the optimal parameters and structures of the ANFIS-based precipitation forecast model, a tabu search was applied to identify the adjacent radius in subtractive clustering when constructing the ANFIS structure. The coupled structure was also employed to establish a precipitation forecast model across short and long lead-times in order to improve the accuracy of long-term precipitation forecasts. The study area is the Shimen Reservoir, and the analyzed period is from 2001 to 2009. Results showed that the optimal initial ANFIS parameters selected by the tabu search, combined with the dual-model hybrid method and the coupled structure, provided the favors in computation efficiency and high-reliability predictions in typhoon precipitation forecasts regarding short to long lead-time forecasting horizons.

1. Introduction

Taiwan is located in the path of typhoons as they move in from the Western Pacific, and as a result, three to five typhoons hit Taiwan annually. Conventional methods used in disaster prevention mainly include engineered structures, but these methods have been proven to be mostly palliative, as typhoons are capable of damaging the hydraulic structures if their associated rainfall and flood discharge exceed the design limit. Therefore, effective methods of ameliorating typhoon-related disasters need to include nonengineered disaster prevention programs, such as effective disaster forewarning and associated response mechanisms, which include the ability to identify the disaster before it occurs. One such method is the regional Quantitative Precipitation Forecast (QPF) information system. In the output of QPF, an accurate forecast of the duration of rainfall and its location is equally as important as the forecasted amount of precipitation. Information related to forecast precipitation can provide the public (and meteorological or water conservation departments) an estimate of expected precipitation in particular catchment areas, in addition to the ability to forewarn of potential debris flow hazards or of the need to employ flood control operations in reservoirs. Therefore, the QPF is crucial for use in disaster forewarning and associated responses.

Due to the powerful capability of artificial neural network (ANNs) to model nonlinear systems without the need to make any assumptions, in recent years ANNs have been used increasingly in applications for modeling hydrological processes, including precipitation forecasts (Govindaraju, [1]; Lin and Wu, [2]). The advantages of using ANNs include the ability to derive accurate short-term forecasts with low building costs, while the disadvantages are (1) unsatisfactory long-term precipitation forecasts with a significant time delay and (2) lower forecast accuracy in relation to higher amounts of precipitation.

Research related to the use of ANNs is summarized as follows. Hsu et al. [3] developed a modified counter propagation artificial neural network for transforming satellite infrared images to precipitation over a watershed, and Kuligowski and Barros [4] developed a feedforward neural network (FFNN) architecture with upper atmospheric wind direction and antecedent observed rainfall data to forecast short-term (0–6 h) precipitation for a specific location. Luk et al. [5] employed backpropagation neural networks (BPNN) in a short-term precipitation forecast in urban catchment areas, focusing on the effects of temporal and spatial information on the precipitation forecast. This study found that there was high forecast accuracy along with a reduced time delay when the input of the neural network included data from a certain number of adjacent rainfall stations. Grecu and Krajewski [6] applied BPNN and a statistical method to radar echo data in QPF and discussed the differences in results arising from the different spatial resolutions and echo sizes. Toth et al. [7] used the linear autoregressive model, BPNN, and nonparametric nearest-neighbors method for rainfall prediction by taking precipitation as the only input of the model; the results showed a significant improvement in long-term precipitation forecast using the ANN. Luk et al. [8] studied the performance of multilayer feedforward neural networks, partial recurrent neural networks, and time delay neural networks in precipitation forecasts within urban catchment areas. Results showed that the three models were all effective in precipitation forecasts and further indicated that precipitation time series were characterized with a short-term memory and that the predicted precipitation presented time delays under large lead-times. Liu et al. [9] proposed a scheme of instantaneous rainfall estimation based on the radial basis function neural network trained with WSR-88D Doppler radar observations. Lin and Chen [10] applied the ANN to typhoon rainfall forecast, and their results showed that precipitation was significantly underestimated when the model input only included typhoon characteristic parameters but that the forecast accuracy was improved by adding appropriate data from adjacent rainfall stations. Valverde Ramírez [11] developed two precipitation forecast models in São Paulo State, Brazil. Their results showed that a better performance can be achieved with the ANN model compared to the multiple linear regressions (MLR) model, particularly when using nonlinear phenomenon such as precipitation forecasting. Hong [12] developed a hybrid model of RNNs (recurrent artificial neural networks) and SVMs (support vector machines), namely, RSVR, to forecast typhoon precipitation, where a chaotic particle swarm optimization algorithm (CPSO) was employed to choose the parameters of the SVR model. The results showed that the proposed model yielded a good forecasting performance. Lin and Wu [2] combined the self-organizing map (SOM) and the multilayer perceptron network (MLPN) to develop a hybrid typhoon precipitation forecast model. In the proposed model, a data analysis technique was developed based on the SOM, which is able to perform cluster analysis and discrimination analysis in one step, and the MLPN is used as a nonlinear regression technique to construct the relationship between the input and output data. The results showed the proposed model to deliver a more precise forecast than the model developed using a conventional neural network approach.

Based on this previous research, there is an evident lack of studies related to instantaneous typhoon precipitation forecasts using ANN techniques. However, the accurate forecast of rainfall during typhoons is extremely difficult because of the variable tracks of typhoons and the complicated interaction between typhoon circulations and diverse terrain (Huang et al., [13]). As a result, the accurate and effective application of ANN techniques (within the field of artificial intelligence) in precipitation forecasts for multiple lead-times during typhoons is a subject that urgently requires development and scientific breakthrough. Typhoon precipitation is characterized by a fuzzy, chaotic, and highly uncertain nonlinear system. The adaptive network-based fuzzy inference system (ANFIS) which is widely used as solution for various real world problems contains fuzzy logic reasoning and learning, and is therefore considered to be a promising candidate technique for use in typhoon precipitation forecasts. Current available literature related to ANNs shows that the selection of network architecture (both input selection parameters and network architecture) and its efficient training are very time consuming, and major obstacles are inherent in its day to day applications (Nasseri et al., [14]). The most commonly used method to retrieve the parameters and structures of ANNs is a random trial-and-error method, which is time consuming and proves difficult in retrieving the optimal parameters, and there is only limited previous research performed on the optimization of parameters and structures of ANNs. Of these researches, El-Fallahi et al. [15] employed the generalized reduced gradient (GRG) method to optimally select initial weights and bias values of BPNN. Results showed that the GRG method was able to effectively search the optimal weights and bias values within a reasonable computing time. In addition, Nasseri et al. [14] established the precipitation forecast model based on FFNN and used the back propagation learning algorithm and the genetic algorithm in network training and optimization, respectively. Results showed the superior performance of the genetic algorithm in precipitation forecasting by selecting optimal network inputs, compared with randomly selected inputs. Furthermore, Leahy et al. [16] applied a simulated annealing algorithm and tabu search to adjust the weights and the number of nodes of the network; results showed that the optimal ANN structures and parameters selected by the simulated annealing algorithm, along with a tabu search, significantly reduced the error and network complexity. In addition, this technique yielded promising forecast results.

However, these above-mentioned studies have not addressed the problems regarding optimization of the parameters and structures of ANFIS. The objective of this study, therefore, is to develop multiple mechanisms and methods for building a typhoon precipitation forecast model based on ANFIS and BPNN, and to compare their respective advantages and disadvantages. Through the study of various methods, which include the optimal selection of the ANFIS parameters and structures based on a merging of tabu search and subtractive clustering, a precipitation forecast for a specific lead-time using a single-model structure and dual-model hybrid structure, and a multiple lead-time precipitation forecast using coupled and noncoupled structures, we aim to improve the modeling efficiency for ANFIS, ameliorate the forecasted ability for high intensity precipitation, and enhance the forecasted accuracy for longer lead-time, respectively.

2. Development of Methodology

2.1. Procedures

The procedures used in this study are divided into four steps as shown in Figure 1. The detailed procedures are thoroughly described as follows.

Figure 1: Flowchart of the methodology.

Step 1. The optimal inputs for the typhoon precipitation forecast model were selected. A nonparametric statistical correlation analysis was applied to data related to meteorological and precipitation factors during previous typhoons. After determining threshold values, the variables with higher correlation coefficients were selected as the model inputs.

Step 2. The optimal construction mechanism of ANFIS was determined (Section 2.3). Fuzzy subtractive clustering approach (Section 2.3.2) was used to determine the number of membership functions in the fuzzy rule base as well as the function parameters during the model construction. The initial adjacent radius in fuzzy subtractive clustering was selected using two methods: the conventional random trial-and-error method and the tabu search algorithm, in order to rapidly, automatically, and accurately retrieve the optimal parameters and structures of ANFIS (Section 2.3.4).

Step 3. The optimal forecast model structure for a specific lead-time was retrieved. Precipitation forecast models based on ANFIS and BPNN were separately constructed using a dual-model hybrid structure for a future period from 1 to 6 hours. The dual-model hybrid structure was created by constructing and combining the higher and lower precipitation models. The single-model structure and dual-model hybrid structure were then compared to determine which model was superior.

Step 4. The optimal structure for the multiple lead-time precipitation forecast model was retrieved. Precipitation forecast models at different lead-times were coupled and compared with the noncoupled structure.

2.2. Selection of Model Inputs

2.2.1. Candidate Predictor

Three types of predictors were used in this study.

(1)Typhoon factor: longitude, latitude, central wind speed, central pressure, grade 7 typhoon radius, grade 10 typhoon radius, and typhoon movement speed.(2)Rainfall station factor: rainfall at the forecasting ground station and rainfall from adjacent stations.(3)Factors calculated from the typhoon factors using fundamental meteorological formulae as follows.

(1)The distance between the typhoon and the forecasting station : this distance can be obtained using a conversion formula from longitude/latitude to distance:

where and are the latitudes of the typhoon center and the forecasting station at time and and are longitudes of the typhoon center and the forecasting station at time .(2)The upper wind speed at the rainfall station : the observed instantaneous grade 7 and grade 10 typhoon speed and radius were used in the present study to calculate the circulation precipitation in typhoon model. The relationship between wind speed and the distance from typhoon center is described by

where is an empirical value that varies with location; is a constant number; and is the distance between the typhoon and the forecasting station at time . The values of and can be solved from the known speed and radius values of grade 7 and grade 10 typhoons. The upper wind speed at the rainfall station is then obtained by substituting the distance between the typhoon and the station into (2).(3)Typhoon characteristic parameter : the typhoon characteristic parameter is calculated from the radius and radial speed of grade 7 and grade 10 typhoons using the following formula:

where and are grade 7 and grade 10 typhoon radii, respectively, at time and and are radial wind speeds of grade 7 and grade 10 typhoon, respectively, at time .

2.2.2. Selection of Model Inputs

In this study, the favorable inputs used in the forecast model use correlation analysis for decision making, and Spearman’s rank correlation coefficient (Spearman, [17]) is adopted as an analysis index. The analysis mechanism used for the correlation depends on the rank relationship of the time series of two variables, and hence this analysis can determine the correlation and suitability of input, regardless of the kind of relationship that exists between the candidate input and output; that is,

where is Spearman’s rank correlation coefficient, is the number of data, is the candidate input of the forecast model (predictor), is the model output also known as the predictant (future precipitation for forecasted lead-time to ), and and are the sort values of and in their individual time series of the variable, respectively.

2.2.3. Assessment Index of Forecast Models

The performance of the forecast models was primarily evaluated using the mean absolute error (MAE) criterion in the present study. The root mean square error (RMSE), correlation coefficient (CC), and were used to verify the suitability of the solution selected by MAE:

where is the forecasted value at time ; is the measured value at time ; is the benchmark time series; and is the number of data.

Smaller values of MAE and RMSE imply a higher accuracy of the forecast model, and larger CC values indicate a closer coupling between the forecasted and measured series. The value enables the effectiveness of the model to be more realistic and for the model to be formularized generally (which is particularly useful for hydrological forecast models). Precipitation is characterized by a continuous time system, in which precipitation at time can be shifted along time units to form a benchmark value at time , and in this way a benchmark series can be constructed. Errors of the forecasted series and the benchmark series were both calculated from the measured values using (7). A negative value of indicates a poorer performance of the forecast model compared with the benchmark series; shows an equivalent performance of the forecast model and the benchmark series; and a positive value means that the forecast model is superior to the benchmark series.

2.3. Establishment of the Optimal ANFIS Construction Mechanism

The most commonly used method for retrieving ANN parameters and structure is merging the random trial-and-error method with conventional parametric searching algorithm, which is used to identify the most appropriate initial structural parameters and train the best data-driven parameters under the random set up structure. However, the random trial-and-error method is time consuming, and the retrieval of the optimal parameters and structure becomes difficult if the initial solutions are set improperly and if the modeled system is enormous. Therefore, the optimization method was employed in the present study to retrieve the optimal parameters and structures of ANFIS. Specifically, the tabu search was first applied to optimize the adjacent radius in the subtractive clustering algorithm, and subtractive clustering was then used to determine the fuzzy membership function parameters and network structures of ANFIS. Finally, the parameters of the consequent layer were estimated by the least squares estimator to complete the ANFIS model. The model identified was tested at a lead-time of in the present study, as shown in Figure 2. ANFIS, subtractive clustering, and the tabu search used in the tests are described in detail below.

Figure 2: Flowchart of the developed methodology for determining the optimal parameter and structure of ANFIS.

2.3.1. Introduction of ANFIS

ANFIS was proposed by Jang (1993) [18] and is based on a fuzzy inference system constructed by combining the self-organization characteristics of a neural network. Hence, ANFIS integrates two algorithms to improve its accuracy and solves for the best parameters by employing capabilities of learning and self-adaption. ANFIS is composed of an input layer, a rule layer, a normalization layer, a consequent layer, and an output layer, as shown in Figure 3. The modeling tool can transform the fuzzy-complex process and phenomenon into artificial logic language that therefore is a potential approach for typhoon precipitation forecast. The computation and transmission of each layer is described as follows.

Figure 3: Structure of an ANFIS.

(1) Input Layer. This layer projects input to a group of fuzzy sets and estimates the values of a group of membership functions. We adopt a group of Gaussian functions as the membership functions, which can be expressed as follows:

where is the membership function, and are the antecedent parameters, is the number of inputs, and is the number of the fuzzy membership functions of input .

(2) Rule Layer. This layer precedes the antecedent match of the fuzzy logic rule between variables and then applies a T-norm product operation to obtain the weighted value of each rule; that is,

where is the weighted value and is the number of rules.

(3) Normalization Layer. The node of this layer computes the output ratio between the node and all other nodes; that is,

(4) Consequent Layer. The output of the consequent layer node is the product of the outputs of the normalization layer and the Sugeno fuzzy model (Takagi and Sugeno, [19]); that is,

where represents the consequent parameters and is equal to 1.

(5) Output Layer. This layer sums the outputs of the previous layer to compute the model output; that is,

ANFIS is a feedforward neural network and is constructed by supervised learning. The network parameters can be divided into antecedent parameters (nonlinear parameters: , ) and consequent parameters (linear parameters: ), and the model structure is determined by setting the number of membership functions in the input layer and the number of nodes of the rule layer. In tradition, the model structure and initial parameters are determined by random trial-and-error method, and the calibrated parameters after data-driven training are retrieved by least square estimator (LSE) and gradient steepest descent method.

To decrease the time for model construction in obtaining the best network structures and parameters, this study constructs ANFIS using hybrid algorithms including subtractive clustering (SC) and a least square estimator (LSE). The input and output vectors were first classified by subtractive clustering before training the model. The number of clusters obtained from the classification was set as the number of membership functions for node fuzzification at the various input layers and the number of nodes of the rule layers, and the th membership function node of each input layer node was connected to the corresponding th rule layer node. After determining the network structures, the center point and standard deviation of each cluster were taken as the initial parameters of the input layer membership functions (Gaussian function). The training data were then fed into the network with the consequent linear parameter set and the antecedent nonlinear parameter set solved by the least squares estimator and the gradient steepest descent method, respectively. The corresponding algorithm flowchart of the model construction is shown in Figure 4; this composite construction mechanism only requires the setting of adjacent radius in subtractive clustering and the subsequent training of network structures and parameters are automatically completed. The network structure significantly reduces the time required to retrieve the optimal number of fuzzy membership functions, number of rules, and network parameters, and the optimal network structure and parameters can be obtained after simply setting the adjacent radius between 0 and 1 (Jang, [18]; Jang et al., [20]).

Figure 4: Flowchart of training the optimal parameters and structure of ANFIS by combining subtractive clustering, least squares estimator, and gradient steepest descent method.

2.3.2. Introduction of Subtractive Clustering

Subtractive clustering was employed in the present study to construct fuzzy if-then rules in order to reduce the number of parameters of the fuzzy membership function in the ANFIS model. This was performed to establish a suitable rule base in the fuzzy inference system. Subtractive clustering was proposed by Chiu [21], in which every data point is treated as the candidate of the cluster center. Subtractive clustering is a fast and independent clustering method: the computational complexity is proportional to the number of data and is independent from the system dimension. For example, are sets of data in an -dimensional space and the corresponding density measures are defined as

where the adjacent radius is a positive number representing the distance near the center, and the data points outside the radius have minimum impact on the density measure. The density measure is calculated for each data point (), and the one with the highest density () is selected as the first cluster center (). The definition of density measure is then modified to select the next cluster center. Assuming that is the cluster center selected at the th round, and the corresponding density measure is , the modified formula is as follows:

where radius has the same definition as and is usually set as so that the selected center will not be too close to that of the previous one. The above procedure of cluster center selection is repeated until a termination condition is reached or there are a sufficient number of cluster centers.

2.3.3. Establishment of the Optimal Model

In this study, the tabu search was used in the present study to optimize the adjacent radius in subtractive clustering to obtain the optimal ANFIS parameters and structures. The objective function minimizing the forecast error is as follows:

A smaller value of the objective function ( value) indicates a higher accuracy of the forecast model. The value of the adjacent radius is confined to between 0 and 1, and the calculations across the layers of ANFIS that all belong to constraints are shown as follows:

where is the adjacent radius. In order to improve the forecast accuracy, single forecast models were constructed for each lead-time with no connections between them and were classified as noncoupled structures, wherein the single-model structure and the dual-model hybrid structure were first compared to determine the subsequent research topic regarding forecast model structure for a specific lead-time (the multiple lead-time precipitation forecast models based on coupled and noncoupled structures are described in Section 2.5).

2.3.4. Introduction of Tabu Search

The tabu search was proposed by Glover [22] and Glover and Laguna [23] and guides the search direction and region using different types of memory. During the search, a search direction or region can be favored or prohibited according to the memory and rules. Additionally, the search can exit at a local optimum region and avoid repeated searches through the definition of a tabu list that includes the type and length of the search variables and the associated objective function value, where every iteration only searches to find the best candidate solution. Hence, this search mechanism is not only able to significantly improve the search efficiency and accuracy but also able to obtain the best global solution. The tabu search was thus applied to optimize the adjacent radius in subtractive clustering, as required in the ANFIS model construction. A diagram of the method is illustrated in Figure 2.

2.4. Retrieval of the Optimal Forecast Model Structure for a Specific Lead-Time

During typhoons, an accurate forecast of higher amounts of precipitation is considered to be much more important than amounts of lower precipitation. According to previous research, precipitation has often been underestimated in higher rainfall, indicating different underlying mechanisms involved in higher and lower precipitation (Luk et al. [5]; Lin and Chen [10]; Lin and Wu [2]). To address this issue, the present study optimized the forecast of higher precipitation to reduce the forecast error. In addition, it was expected that the forecast error of higher precipitation would be lower than that obtained from the single-model structure.

According to the Central Weather Bureau in Taiwan, a heavy rainfall is defined as 24-hour accumulated rainfall of 50 mm or above, during which time there is at least one hour where precipitation amounts to 15 mm or above. In this study therefore, high precipitation was defined as an amount of 15 mm/hr or above. The dual-model hybrid method proposed in the present study constructs both lower and higher precipitation forecast models that can be subsequently selected within instantaneous forecasting. The lower precipitation model is usually initially applied, and if the forecasted precipitation is less than 15 mm/hr, the forecast value is directly accepted. However, if the forecasted precipitation is greater than 15 mm/hr, the higher precipitation model is used instead to obtain the corresponding forecast value. In this section, the single-model method and the dual-model hybrid method are first compared within noncoupled structures in order to determine the model construction for a specific lead-time in the combined multiple lead-time forecast model, based on noncoupled and coupled structures (see Section 2.5).

2.5. Combined Multiple Lead-Time Precipitation Forecast Model Based on Coupled and Noncoupled Structures

Due to the complexity and uncertainty of the relationship between the atmosphere and physiography, the accuracy of any precipitation forecast decreases with any increase in the forecast lead-time. In this present study, two structures are proposed to improve the accuracy of long-term precipitation forecasts by constructing a multiple lead-time precipitation forecast model. The first structure is a noncoupled model, where the inputs of the forecast models for different lead-times are all instantaneous observed data, and the outputs of forecast models at each lead-time are not coupled with the model inputs at the next lead-time. The second structure is a coupled model, which better characterizes the relationship between series and provides enhanced forecasting information. The coupled structure is therefore expected to provide a more accurate precipitation forecast for a longer lead-time. Observed information includes the typhoon factors that are input and the meteorological and physical factors, whereas the precipitation factor couples both observed and forecast information. Taking the present study as an example, forecast precipitation at is required for the coupled model, where the forecast precipitation at replaces the input precipitation at in the model, the forecast precipitation at replaces the input precipitation at in the model, and so on. The noncoupled and coupled structures are shown in Figure 5. After model training, it will then be intended to investigate the performance of the two structures in practical applications using model verification and assessment indices from the testing stage.

Figure 5: Noncoupled and coupled structures for precipitation forecast regarding short to long lead-time forecasting horizons.

2.6. Introduction of BPNN

The most commonly used BPNN was employed in the present study to construct models based on a noncoupled single-model and dual-model hybrid methods, as well as a coupled structure. The BPNN-based forecast performance was then compared with that of the newly developed ANFIS model. BPNN, developed by Rosenblatt [24], is a single-layer perceptron with no hidden layers and is therefore incapable of solving certain problems. The conception of a multilayer neural network was not realized until the well-known back propagation learning method was proposed by Rumelhart and McClelland [25], which developed an effective strategy for solving large amount of node weights.

The BPNN structure is composed of multilayer perceptron (MLP) and generally employs the error back propagation (EBP, also abbreviated as back propagation (BP)) algorithm as the learning algorithm. The combination of MLP and EBP is known as BPNN. The BP algorithm is classified as a multilayer FFNN, and the nonlinear mapping between inputs and outputs is handled by supervised learning. The commonly adopted BPNN structure is shown in Figure 6 and includes an input layer, a hidden layer, and an output layer. The input and output layers represent the input and output values with the number of nodes determined by the type of problem to be solved. The number of hidden nodes is generally retrieved using the trial-and-error method. In addition, the number of hidden layers can be increased from one to multiple layers based on the complexity of the problem. The nodes of different layers are connected by related weights. The input values are directly transmitted to the hidden layer through the input layer and converted into the outputs by the activation functions after weighted summation. The output values are fed back into the input layer in the same manner. In BPNN, the output of the th node in the th layer is the conversion function of the node outputs in the th layer:

where represents the th node output in the th layer, which is the input for the first layer. is the activation function and is the weighted summation of the outputs from the th layer:

where is the node weight connecting the th node in the th layer and the th node in the th layer and is the bias value of the th node in the th layer. In the present study, the activation functions at the BPNN hidden layer and output layer were set as the hyperbolic function and the linear activation function, respectively. As a supervised learning network, the BPNN aims to reduce the difference between network outputs and objective outputs. The objective function (i.e., cost function) is defined as follows:

where is the target output value of the th node and is the network output value of the th node. Therefore, the network learning process is a process that minimizes the objective function . In the present study, the optimization of BPNN parameters was handled by the conjugate gradient method to search for the minimum value of the objective function .

Figure 6: Structure of a BPNN.

3. Application

3.1. Study Area

The methodology proposed in the present study was applied to the Shimen Reservoir catchment area, which measures approximately 763.4 km². The main stream within this area is the Dahan Creek, which is the upper stream of the Tamsui River. Rainfall stations in the Shimen Reservoir catchment area used in the present study were set up by the Northern Region Water Resources Office at the Water Resources Agency of Ministry of Economic Affairs and include locations in Shimen, Hsia-Yun, Gao-Yi, Ba-Ling, Galahe, Yu-Feng, Bai-Shi, Zhenxibao, Siciouish Mountain, and Chi-Duan (Figure 7).

Figure 7: Study area and the distribution of rainfall gauges.

3.2. Data Used in Model Construction

This study used instantaneous observed information, such as rainfall and typhoon characteristics, to forecast precipitation for six lead-times from to . The output variables were taken as the precipitation forecast of the two rainfall stations of to in the Shimen Reservoir catchment area: Yu-Feng and Hsia-Yun stations. A total of 25 typhoons were selected in the present study based on the criterion that a typhoon land warning was issued and the corresponding cumulated precipitation was greater than 50 mm. The model construction included three stages, namely, a training, validation, and testing stage (Table 1).

Table 1: The adopted typhoon events among training, validation, and testing stage.

Typhoon name
Time period of typhoon warning
Total precipitation (mm)
Data number
Total data number

Training
Utor
2001/07/03~2001/07/05
92
46
804

Toraji
2001/07/28~2001/07/31
51
75

Lekima
2001/09/24~2001/09/28
167
91

Haiyan
2001/10/15~2001/10/16
150
27

Sinlaku
2002/09/05~2002/09/08
355
73

Haitang
2005/07/16~2005/07/20
399
75

Matsa
2005/08/04~2005/08/06
1271
54

Talim
2005/08/30~2005/09/01
328
51

Khanun
2005/09/10~2005/09/11
132
27

Sepat
2007/08/16~2007/08/19
218
66

Wipha
2007/09/17~2007/09/19
504
42

Krosa
2007/10/05~2007/10/07
709
63

Fung-Wong
2008/07/27~2008/07/29
120
57

Jangmi
2008/09/27~2008/09/29
249
57

Validation
Nari
2001/09/15~2001/09/19
937
111
421

Dujuan
2003/08/31~2003/09/02
80
42

Rananim
2004/08/11~2004/08/12
424
36

Aere
2004/08/23~2004/08/26
1559
69

Nock-Ten
2004/10/24~2004/10/25
286
42

Nanmadol
2004/12/03~2004/12/04
115
24

Longwang
2005/10/01~2005/10/03
172
51

Kalmaegi
2008/07/17~2008/07/18
67
46

Testing
Mindulle
2004/06/29~2004/07/02
74
72
252

Sinlaku
2008/09/12~2008/09/15
1084
87

Morakot
2009/08/06~2009/08/10
493
93

3.3. Results and Discussion

3.3.1. Model Inputs Selection

Correlation analysis was applied in the present study to assess the correlation coefficient between each input factor and the precipitation at each lead-time for each rainfall station. The selected model inputs and corresponding correlation coefficients for Yu-Feng station and Hsia-Yun station are shown in Tables 2 and 3, respectively. Research conducted by Lin and Chen [10] revealed that excessive model inputs could introduce additional noise into the model, therefore 10 input factors were selected based on the correlation coefficients and subsequent tests were used to avoid excessive inputs while retaining the forecasting information necessary for the model. In addition, we aimed to include certain typhoon factors as the model inputs in order to supplement information (Lin and Chen [10]; Lin and Wu [2]) that could not be represented by the rainfall stations, and therefore three typhoon factors and seven rainfall station factors were selected as the inputs. The results showed that the forecast factor mostly used was precipitation information, and that typhoon factors and calculated physical factors were also indispensable. These parameters are considered to be essential for use in eliminating the time lag and for characterizing rainfall.

Table 2: The selected model inputs and corresponding correlation coefficients for Yu-Feng station.

Forecasted lead-time
+ 1
+ 2
+ 3
+ 4
+ 5
+ 6

Typhoon central longitude
X
0.450
0.472
0.492

Typhoon central maximum wind speed
0.436
0.439
0.443
0.446
0.45
0.453

Typhoon central atmospheric pressure
0.451
0.452
0.457
0.46
0.465
0.469

The upper wind speed at Yu-Feng station
0.46
0.454
0.432
X

Hsia-Yun precipitation at time
0.894
0.867
0.819
0.819
0.795
0.773

Gao-Yi precipitation at time
0.908
0.882
0.831
0.831
0.809
0.788

Ba-Ling precipitation at time
0.895
0.869
0.820
0.820
0.799
0.778

Zhenxibao precipitation at time
0.884
0.858
0.812
0.812
0.791
0.761

Bai-Shi precipitation at time
0.895
0.872
0.819
0.819
0.797
0.772

Yu-Feng precipitation at time
0.917
0.89
0.839
0.839
0.819
0.792

Yu-Feng precipitation at time   −  1
0.890
0.871
0.822
0.822
0.797
0.777

Table 3: The selected model inputs and corresponding correlation coefficients for Hsia-Yun station.

+ 1
+ 2
+ 3
+ 4
+ 5
+ 6

Typhoon central longitude
X
0.423
0.447
0.465

Typhoon central maximum wind speed
X
0.415
X
0.424
0.428

Typhoon central atmospheric pressure
0.419
0.423
0.429
0.434
0.439
0.444

Distance between observed station and typhoon center
0.437
0.419
X

The upper wind speed at Hsia-Yun station
0.483
0.467
0.449
0.432
X

Gao-Yi precipitation at time
0.857
0.825
0.809
X
0.772

Ba-Ling precipitation at time
0.896
0.863
0.846
0.825
0.806
0.791

Gao-Yi precipitation at time
0.872
0.844
0.824
0.809
0.794
0.776

Yu-Feng precipitation at time
0.884
0.857
0.838
0.825
0.806
0.793

Zhenxibao precipitation at time
X
0.798
0.784
X

Bai-Shi precipitation at time
0.864
0.838
0.824
0.806
0.791
0.777

Hsia-Yun precipitation at time   −  1
0.872
0.84
0.818
0.802
0.78
0.764

Hsia-Yun precipitation at time
0.912
0.872
0.842
0.819
0.803
0.783

3.3.2. Determination of the Optimal ANFIS Construction Mechanism

The present study combined a tabu search and subtractive clustering to determine the optimal structures and parameters of ANFIS. The adjacent radius in subtractive clustering was optimized using two methods: the traditional random trial-and-error method and the tabu search. The MAE values of the forecasted precipitation at during validation were used to compare the two methods and to identify the superior model. The 10 inputs and one output (11 variables in total) were included in the proposed model, and, therefore, the number of decision variables for the adjacent radius was set as 11.

The random trial-and-error method generated 11 random numbers between 0 and 1 each time for the value of adjacent radius in subtractive clustering, followed by model parameter training using LSE and the gradient steepest descent method. A total of 1000 models were generated and trained, and the one with the smallest MAE value was selected as the optimal model in the random trial-and-error method. The tabu search first generated 50 sets of random solutions for training and then selected the one with the smallest MAE value as the initial solution of the tabu search. The optimal solution was then obtained by conducting an additional 20 iterations of the tabu search, with a step shift of the decision variables set at 0.05.

The adjacent radius in subtractive clustering was found using the tabu search, and validation of the MAE values of the precipitation forecast model at at Yu-Feng station and Hsia-Yun station is illustrated in Figure 8. These figures show that the MAE values of the constructed forecast models significantly decreased with an increase in the number of iterations. In addition, the optimal MAE values were markedly lower than those from the random trial-and-error method. Furthermore, the tabu search required only 490 iterations, which was less than half of the computation amount and time required by the random trial-and-error method. The testing results shown in Table 4 also indicate that the tabu search was able to retrieve the optimal ANFIS parameters and structures more efficiently than the random trial-and-error method. The lower testing values of MAE and RMSE and higher testing value of CC and received from the tabu search approach represent that the constructed ANFIS-based precipitation forecast model using the tabu search approach is more accurate, stable, and effective than the random trial-and-error method. Therefore, the tabu search was selected for determining the optimal adjacent radius.

Table 4: Comparison of the value of assessment indices by applying tabu search and trial-and-error method for retrieving the optimal parameter and structure of ANFIS on testing stage.

Testing value of random trial-and-error method
Testing value of tabu search

Yu-Feng station
Hsia-Yun station
Yu-Feng station
Hsia-Yun station

MAE
3.84
3.944
3.798
3.532

RMSE
6.834
6.329
6.626
6.075

CC
0.682
0.651
0.709
0.682

0.185
0.089
0.234
0.161

(a)

(b)

Figure 8: Comparison of MAE of validation between tabu search approach and random trial-and-error method for construction of ANFIS-based precipitation forecast model: (a) Yu-Feng station; (b) Hsia-Yun station.

3.3.3. Retrieval of Optimal Forecast Model Structure for a Specific Lead-Time

(1) ANFIS-Based Single and Hybrid Precipitation Forecast Model for a Specific Lead-Time. In this section, the noncoupled single-model structure and the dual-model hybrid structure were compared for each lead-time. The construction method used for the optimal model was the same as that used for constructing the coupled model for each lead-time. The forecast performance of the single and hybrid models at Yu-Feng and Hsia-Yun stations are compared in Figures 9 and 10. Based on the lower MAE value of assessment index of dual-model hybrid structure than single-model, the results show that the hybrid model effectively improves the forecast accuracy especially for high intensity precipitation by between and . The validated and testing results of Yu-Feng station and Hsia-Yun station all can be verified. However, lower forecast accuracy at Yu-Feng station at was observed because the forecast at a greater lead-time was prone to a time lag, and when using a combination of the higher and lower precipitation models, the precipitation forecast by the lower precipitation model was actually forecast by the higher precipitation model, which increased the forecast error. Based on test results, the dual-model hybrid structure of the ANFIS forecast model outperformed that of the single-model structure and was therefore selected to construct the coupled forecast model for multiple lead-times.

(a)

(b)

Figure 9: Comparison of the forecast performance of the single and hybrid ANFIS-based models at Yu-Feng station: (a) validation; (b) testing.

(a)

(b)

Figure 10: Comparison of the forecast performance of the single and hybrid ANFIS-based models at Hsia-Yun station: (a) validation; (b) testing.

(2) BPNN-Based Single and Hybrid Precipitation Forecast Models for a Specific Lead-Time. The BPNN-based single model was constructed using one model at each forecasted lead-time . The number of hidden nodes was identified using the following procedure: node numbers from 2 to 10 were first separately trained for 10 times with 5000 iterations each time. The three node numbers giving the smallest average MAE values were then selected to receive an additional 1000 iterations of training. Finally, the node number with the smallest MAE value was determined as the optimal model for a specific lead-time. A comparison of the validation and testing of MAE values at Yu-Feng and Hsia-Yun stations showed that the performance of the BPNN-based hybrid and single models were comparable, as shown in Figures 11 and 12. However, when the hybrid model outperformed the single model, the MAE values of the hybrid model were much smaller than those of the single model. Therefore, the dual-model hybrid structure was selected to construct the coupled multiple lead-time forecast model.

(a)

(b)

Figure 11: Comparison of the forecast performance of the single and hybrid BPNN-based models at Yu-Feng station: (a) validation; (b) testing.

(a)

(b)

Figure 12: Comparison of the forecast performance of the single and hybrid BPNN-based models at Hsia-Yun station: (a) validation; (b) testing.

3.3.4. Retrieval of the Optimal Precipitation Forecast Model Structure for Multiple Lead-Times

In this section, four multiple lead-time forecast models based on the hybrid model structure were compared (BPNN-noncoupled, BPNN-coupled, ANFIS-noncoupled, and ANFIS-coupled). In addition to the validation and testing of the MAE value, the efficiency and convenience of the model construction were also considered as criterions for model performance assessment. The validation MAE values of the models at multiple forecast lead-times at Yu-Feng and Hsia-Yun stations are shown in Figure 13, and the testing index values of the models at Yu-Feng station and Hsia-Yun station are shown in Figures 14 and 15, respectively. The results showed that the ANFIS-coupled model almost consistently outperformed the ANFIS-noncoupled model at both Yu-Feng and Hsia-Yun stations. In particular, the ANFIS-coupled model achieved a greater improvement of forecast accuracy at a greater lead-time based on the values of MAE, RMSE, CC, and , indicating a true improvement of the ANFIS-coupled model at a larger lead-time. However, changes in the forecast performance were insignificant between the BPNN-coupled and BPNN-noncoupled models, implying that the BPNN precipitation forecast model based on the coupled structure was no better than the one based on the noncoupled structure for multiple lead-time forecasting. It represents that because the simulated mechanism of ANFIS owns the higher ability of tolerance and adaption for the transmitted error based on the fuzzy-adaptive routing structure, the coupled structure of ANFIS can perform better forecasted accuracy, stability, and effectiveness than BPNN.

(a)

(b)

Figure 13: The validation MAE values of the models at multiple forecast lead-times: (a)Yu-Feng station; (b) Hsia-Yun station.

(a)

(b)

Figure 14: The testing index values of the coupled and noncoupled models at Yu-Feng station: (a) MAE and RMSE; (b) CC and .

(a)

(b)

Figure 15: The testing index values of the coupled and noncoupled models at Hsia-Yun station: (a) MAE and RMSE; (b) CC and .

The number of nodes in hidden layer of the optimal BPNN-noncoupled precipitation forecast model for Yu-Feng station from lead-time 1 hour to 6 hours is 8, 8, 9, 9, 9, and 9, respectively, and, for Hsia-Yun station, is 8, 8, 9, 9, 10, and 10, respectively. In BPNN-noncoupled forecast model, the optimal activation function of hidden layer for shorter lead-time 1 hour to 3 hours is hyperbolic function, but, for longer lead-time 4 hours to 6 hours, the optimal activation function is linear function. It represents that the real-time observed meteorological rainfall information can retrieve the future precipitation mechanism, and linear activation function can better solve the problem of underestimating future precipitation in longer lead-time. Moreover, the number of rules derived from the optimal ANFIS-coupled precipitation forecast model for Yu-Feng station from lead-time 1 hour to 6 hours is 6, 7, 7, 8, 8, and 8, respectively, and for Hsia-Yun station is 7, 8, 8, 8, 9 and 9, respectively. The optimal model structure no matter ANFIS or BPNN all would be getting more complex along the increase of forecasted lead-time both for Yu-Feng station and for Hsia-Yun station. It represents that because the rainfall uncertainty of longer forecasted lead-time is too high, the forecast model including ANFIS and BPNN needs more rules and network connections, respectively, to retrieve the typhoon-rainfall mechanism. Furthermore, because the location of Hsia-Yun station is more near at marginal basin than Yu-Feng station, under the effect of terrain and lower representative information offered by adjacent observation station, the forecast model for Hsia-Yun station including ANFIS and BPNN needs more complex rules and network structures than Yu-Feng station, respectively, to retrieve the typhoon-rainfall mechanism.

From the perspective of model construction time, the ANFIS model was found to be significantly better than the BPNN model. Based on the computer computation in the present study, it took BPNN approximately 40000 seconds to construct a forecast model for a single lead-time, while the ANFIS only required approximately 1800 seconds. This means that without parallel computation using multiple computers, BPNN will require three days to complete the model construction from to , and ANFIS will only require three hours. As a result, the ANFIS-based precipitation forecast model is considered to be superior based on its construction speed and convenience that can solve the time-consuming problem. Additionally, the overall performance of the models at each forecast lead-time presented little difference, and most of the validation and testing results of the ANFIS-coupled model were close to optimum. Therefore, from the perspective of stability and reliability, the ANFIS-coupled model structure is a more favorable choice for typhoon precipitation forecast at multiple lead-times. The validation and testing precipitation forecast results at of the ANFIS-based noncoupled hybrid model at Yu-Feng and Hsia-Yun stations are shown in Figures 16 and 17, respectively, and those of are shown in Figures 18 and 19, respectively.

(a)

(b)

Figure 16: The validation and testing precipitation forecast results at of the ANFIS-based hybrid dual-model at Yu-Feng station: (a) validation; (b) testing.

(a)

(b)

Figure 17: The validation and testing precipitation forecast results at of the ANFIS-based hybrid dual-model at Hsia-Yun station: (a) validation; (b) testing.

(a)

(b)

Figure 18: The validation and testing precipitation forecast results at of the ANFIS-based coupled hybrid dual-model at Yu-Feng station: (a) validation; (b) testing.

(a)

(b)

Figure 19: The validation and testing precipitation forecast results at of the ANFIS-based coupled hybrid dual-model at Hsia-Yun station: (a) validation; (b) testing.

4. Conclusion

Typhoon precipitation is characterized as a chaotic, fuzzy, highly uncertain, and nonlinear system. The adaptive network-based fuzzy inference system (ANFIS) has the ability to reason and learn using fuzzy logic. Therefore, the present study applied ANFIS, combined with multiple artificial intelligence techniques, to develop the optimal construction mechanism using model parameters and structures for a typhoon precipitation forecast model of one- to six-hour lead-time. The proposed system was evaluated by a comparison with the most commonly used back propagation neural networks (BPNN). The present study first employed nonparametric correlation analysis to assess the correlation between multiple observed and forecasted variables (typhoon factors, calculated physical factors, and precipitation information from adjacent rainfall stations) and the precipitation of the forecasting station, so that the most appropriate input variables at each lead-time were identified. In order to effectively and accurately retrieve the optimal parameters and structure of the ANFIS-based precipitation forecast model, the tabu search was applied to identify the adjacent radius in subtractive clustering when constructing the ANFIS structure. The results of the tabu search were further compared with the random trial-and-error method to obtain a fast and automatic solution. To improve the forecast accuracy of higher precipitation, the single-model method and the dual-model hybrid method were used in the present study to forecast higher and lower precipitation for a specific lead-time. In addition, coupled structures were adopted for the construction of multiple lead-time precipitation forecast model to improve the accuracy of long-term precipitation forecast. Lastly, four forecast models for multiple lead-times were compared, including ANFIS-coupled, ANFIS-noncoupled, BPNN-coupled, and BPNN-noncoupled.

The proposed method was applied to the Taiwan Shimen Reservoir catchment area with a study period from 2001 to 2009. The results showed that (1) subtractive clustering was able to effectively simplify the construction and algorithm parameters of ANFIS and the tabu search was able to effectively optimize the parameters in subtractive clustering and hence retrieve the optimal ANFIS parameters and structures. Compared with the random trial-and-error method, the tabu search significantly provided the favors in computation efficiency and high-reliability predictions in typhoon precipitation forecasts. (2) Due to different precipitation mechanisms occurring regarding short to long lead-time forecasting horizons, the present study combined the forecast results from higher and lower precipitation models and predicted typhoon precipitation for a specific lead-time based on the current precipitation level. The results showed that the dual-model hybrid method is superior to the single-model structure in overall MAE and forecasting accuracy during maximum precipitation. (3) The ANFIS model not only outperformed the BPNN model in model construction efficiency, but its forecast accuracy was also equal to that of the BPNN model. The ANFIS model showed a favorable performance in characterizing the highly complex nonlinear relationship between atmosphere and precipitation during typhoons. Therefore, the use of coupling between the inputs and outputs of the ANFIS forecast model at different lead-times was able to effectively improve the accuracy of long-term precipitation forecasts. The forecast performance barely changed between the BPNN-coupled and BPNN-noncoupled structures for multiple lead-times forecasting, indicating that the BPNN model was relatively insensitive to updated precipitation values, and that the error tolerance of the BPNN algorithm mechanism was relatively low.

Conflict of Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.

Acknowledgments

This research was partially supported by the National Science Council, Taiwan (Grant no. NSC 102-2625-M-002-005). In addition, the authors are indebted to the reviewers for their valuable comments and suggestions.

References

R. S. Govindaraju, “Artificial neural networks in hydrology. I: preliminary concepts,” Journal of Hydrologic Engineering, vol. 5, no. 2, pp. 115–123, 2000.

G.-F. Lin and M.-C. Wu, “A hybrid neural network model for typhoon-rainfall forecasting,” Journal of Hydrology, vol. 375, no. 3-4, pp. 450–458, 2009.

K. L. Hsu, X. Gao, S. Sorooshian, and H. V. Gupta, “Precipitation estimation from remotely sensed information using artificial neural networks,” Journal of Applied Meteorology, vol. 36, no. 9, pp. 1176–1190, 1997.

R. J. Kuligowski and A. P. Barros, “Experiments in short-term precipitation forecasting using artificial neural networks,” Monthly Weather Review, vol. 126, no. 2, pp. 470–482, 1998.

K. C. Luk, J. E. Ball, and A. Sharma, “A study of optimal model lag and spatial inputs to artificial neural network for rainfall forecasting,” Journal of Hydrology, vol. 227, no. 1–4, pp. 56–65, 2000.

M. Grecu and W. F. Krajewski, “A large-sample investigation of statistical procedures for radar based short-term quantitative precipitation forecasting,” Journal of Hydrology, vol. 239, no. 1–4, pp. 69–84, 2000.

E. Toth, A. Brath, and A. Montanari, “Comparison of short-term rainfall prediction models for real-time flood forecasting,” Journal of Hydrology, vol. 239, no. 1–4, pp. 132–147, 2000.

K. C. Luk, J. E. Ball, and A. Sharma, “An application of artificial neural networks for rainfall forecasting,” Mathematical and Computer Modelling, vol. 33, no. 6-7, pp. 683–693, 2001.

H. Liu, V. Chandrasekar, and G. Xu, “An adaptive neural network scheme for radar rainfall estimation from WSR-88D observations,” Journal of Applied Meteorology, vol. 40, no. 11, pp. 2038–2050, 2001.

G.-F. Lin and L.-H. Chen, “Application of an artificial neural network to typhoon rainfall forecasting,” Hydrological Processes, vol. 19, no. 9, pp. 1825–1837, 2005.

M. C. Valverde Ramírez, H. F. de Campos Velho, and N. J. Ferreira, “Artificial neural network technique for rainfall forecasting applied to the São Paulo region,” Journal of Hydrology, vol. 301, no. 1–4, pp. 146–162, 2005.

W.-C. Hong, “Rainfall forecasting by technological machine learning models,” Applied Mathematics and Computation, vol. 200, no. 1, pp. 41–57, 2008.

J.-C. Huang, C.-K. Yu, J.-Y. Lee, L.-W. Cheng, T.-Y. Lee, and S.-J. Kao, “Linking typhoon tracks and spatial rainfall patterns for improving flood lead time predictions over a mesoscale mountainous watershed,” Water Resources Research, vol. 48, no. 9, Article ID W09540, 2012.

M. Nasseri, K. Asghari, and M. J. Abedini, “Optimized scenario for rainfall forecasting using genetic algorithm coupled with artificial neural network,” Expert Systems with Applications, vol. 35, no. 3, pp. 1415–1421, 2008.

A. El-Fallahi, R. Martí, and L. Lasdon, “Path relinking and GRG for artificial neural networks,” European Journal of Operational Research, vol. 169, no. 2, pp. 508–519, 2006.

P. Leahy, G. Kiely, and G. Corcoran, “Structural optimisation and input selection of an artificial neural network for river level prediction,” Journal of Hydrology, vol. 355, no. 1–4, pp. 192–201, 2008.

C. Spearman, “The proof and measurement of association between two things,” The American Journal of Psychology, vol. 100, no. 3-4, pp. 441–471, 1987.

J.-S. R. Jang, “ANFIS: adaptive-network-based fuzzy inference system,” IEEE Transactions on Systems, Man and Cybernetics, vol. 23, no. 3, pp. 665–685, 1993.

T. Takagi and M. Sugeno, “Derivation of fuzzy control rules from human operator's control actions,” in Proceedings of the IFAC Conference on Fuzzy Information, vol. 1, pp. 55–60, Marseille, France, 1983.

J. S. R. Jang, C. T. Sun, and E. Mizutani, Neuro-Fuzzy and Soft Computing: A Computational Approach to Learning and Machine Intelligence, Matlab Curriculum Series, Prentice Hall, Upper Saddle River, NJ, USA, 1997.

S. L. Chiu, “Fuzzy model identification based on cluster estimation,” Journal of Intelligent and Fuzzy Systems, vol. 2, no. 3, pp. 267–278, 1994.

F. Glover, “Future paths for integer programming and links to artificial intelligence,” Computers & Operations Research, vol. 13, no. 5, pp. 533–549, 1986.

F. Glover and M. Laguna, Tabu Search, Kluwer Academic Publishers, Boston, Tex, USA, 1997.

F. Rosenblatt, “The perceptron: a probabilistic model for information storage and organization in the brain,” Psychological Review, vol. 65, no. 6, pp. 386–408, 1958.

D. E. Rumelhart and J. L. McClelland, Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vol. 1, MIT Press, Cambridge, Mass, USA, 1986.