An Artificial Intelligence Approach for Groutability Estimation Based on Autotuning Support Vector Machine

Permeation grouting is a commonly used approach for soil improvement in construction engineering.Thus, predicting the results of grouting activities is a crucial task that needs to be carried out in the planning phase of any grouting project. In this research, a novel artificial intelligence approach—autotuning support vector machine—is proposed to forecast the result of grouting activities that employ microfine cement grouts. In the new model, the support vector machine (SVM) algorithm is utilized to classify grouting activities into two classes: success and failure. Meanwhile, the differential evolution (DE) optimization algorithm is employed to identify the optimal tuning parameters of the SVM algorithm, namely, the penalty parameter and the kernel function parameter. The integration of the SVM and DE algorithms allows the newly established method to operate automatically without human prior knowledge or tedious processes for parameter setting. An experiment using a set of in situ data samples demonstrates that the newly established method can produce an outstanding prediction performance.


Introduction
In construction engineering, permeation grouting is the process that involves the injection of suitable particulate grouts or chemical solutions into the geomaterial with the aim of improving its mechanical properties and reducing the water movement through soils [1].In particular for underground construction works, the inflow of groundwater has always been a substantial challenge for geotechnical engineers [2].Water inflows often cause construction delays and severe damages to the structure quality.Consequently, the grouting activity is an essential task which needs to be performed in a majority of underground construction projects.
Recently, microfine cement grouts have been increasingly employed by geotechnical engineers.The reason is that microfine cement grouts can provide an improved groutability for the target geomaterial and they do not contaminate the surrounding environment.In addition, these grouts are proven to have the capacity of filling cracks with small openings as well as penetrating fine soils with very low permeability [3].
Nonetheless, one of the main challenges in the utilization of microfine cement grouts is how to accurately estimate the groutability of the target geomaterial [4].It is because the grouting process is based on the complex time-dependent transport process of cement grains through the soil matrix.Moreover, besides the grain size of the soil and the grout, other factors that affect the outcome of grouting activities should be taken into account.Due to such complexity, existing empirical formulas [5][6][7][8] can hardly attain satisfactory results.The reason is that these formula-based approaches, which are mostly based on the grain size of the soil and the grout, are unreliable for seminanometer scale grouts.
Experimental studies done by Akbulut and Saglamer [9] and Ozgurel and Vipulanandan [10] found that, in addition to the grain size of the soil and the grout, the water-tocement ratio of grout (w/c), the void size in soil, and the fines content (FC) of the total soil should be considered.Liao et al. [11] pointed out that information of soil gradation, namely, the coefficient of uniformity (  ), which measures the particle size range, and the coefficient of gradation (  ), which characterizes the particle size curve, can be useful for estimating groutability.
Characteristics of construction projects are highly uncertain and intrinsically context-dependent; therefore, artificial intelligence (AI) methods can provide feasible alternatives for groutability prediction.From the perspective of AI, the problem at hand can be modeled as a classification task that contains two class labels (success and failure).Based on the collected data, an AI based approach can be constructed and utilized to classify new input samples.
Artificial neural network (ANN) has been applied to deal with groutability prediction as well as with other problems in the construction industry [4,[11][12][13].Although ANN has been proven to be feasible in the task of groutability estimation, its implementation suffers from several drawbacks.The approach has difficulties in selecting a large number of controlling parameters (the number of hidden layers, the number of neurons in each layer, and the learning rate) [14].Furthermore, one major disadvantage of ANN is that its training process is achieved through a gradient descent algorithm on the error space, which can be very complex and may contain many local minima [15].Thus, the training process is likely to be trapped into a local solution and this undoubtedly hinders the predictive performance.
Recently, support vector machine (SVM), proposed by Vapnik [16], has been applied to resolve a wide span of classification problems.SVM classifies data with several class labels by identifying a set of support vectors from the set of training data; these support vectors have the role in determining the class decision boundary.Although various studies have indicated the superior performance of SVM over ANN [17][18][19], none of previous research works has evaluated the potentiality of SVM for groutability estimation.Thus, this paper is an attempt to fill this gap.
Moreover, when using SVM, the users need to determine its tuning parameters, namely, the penalty and kernel function parameters.Proper settings of these parameters make certain of SVM prediction accuracy.Thus, this research proposes to fuse SVM and differential evolution (DE)-a fast and effective evolution optimization technique-to construct a novel approach for groutability prediction employing microfine cement grouts.In this hybrid mechanism, the SVM technique is used to derive the decision boundary for predicting the consequence of a grouting process.Meanwhile, the DE algorithm is employed to search for the optimal set of SVM tuning parameters.
The remaining part of this paper is organized as follows.The second section of this paper presents the research methodology including the SVM algorithm, the DE algorithm, and the historical data of grouting cases.The framework of the proposed approach is depicted in the third section.The fourth section demonstrates the experimental results.The conclusion of this study is stated in the final section.

Formula-Based Approaches for Groutability Estimation.
In the literature, various researchers have attempted to develop groutability estimation methods by evaluating the relationships between the grain size of the soil and the particle size of the cement.This section of the paper reviews existing groutability prediction approaches which are stated in the form of formulas.
Burwell [6] proposed two formulas to estimate groutability as follows: where  15 and  10 denote the diameters through which 15% and 10% of the total soil mass pass, respectively. 85 and  95 are the diameters through which 85% and 95% of the total grout pass, respectively.
According to [6], it is possible for the grouting process to be successful if  1 is greater than 25 and  2 is greater than 11.Meanwhile, the grouting process is not feasible if  1 is less than 11; if 11 <  1 < 25, the outcome is undefined.
Incecik and Ceren [5] suggested an alternate equation as follows: where  10 denotes the diameter through which 10% of the total soil mass passes. 90 is the diameter through which 90% of the total grout passes.Based on (3), the grouting process succeeds if  is greater than 10.Krizek et al. [7] suggested the identical equations to determine  1 (see (1)) and  2 (see (2)) as [6], but the authors proposed different thresholds: the grouting process succeeds if  1 is greater than 15 and  2 is greater than 8.

Support Vector Machine.
The SVM principles are based on the structural risk minimization and statistical learning theory [16].In general, the SVM technique classifies a data sample by mapping the data points into a high-dimensional feature space and identifying the classification boundary in such space.The advantages of SVM include strong inference capacity, good generalization, fast learning, and accurate prediction [19,20].This section describes the formulation of the SVM algorithm.
Given a training data set {  ,   }  =1 with input data   ∈   and corresponding class labels   ∈ {−1, +1}, the SVM formulation for classification is stated as follows: where  ∈   is the normal vector to the classification hyperplane and  ∈  is the bias;   > 0 are called slack variables;  denotes a penalty constant; and () represents a nonlinear mapping from the input space to the highdimensional feature space.The Lagrangian is given by:  (, , ; ; V) where   ≥ 0, V  ≥ 0 are Lagrange multipliers for  = 1, 2, . . ., .The conditions for optimality are given by After replacing ( 6) in ( 5), we obtain the following dual quadratic programming problem: And the kernel function is applied as follows: The resulting SVM model for classification is expressed as follows: where SV denotes the number of support vectors which are training data points corresponding to   ̸ = 0.The kernel function that is often utilized is radial basis function (RBF) kernel.Description of RBF kernel is given as follows: where  is the kernel function parameter.
In the case of the RBF kernel, there are two tuning parameters (, ) that need to be determined to establish the SVM prediction model.The penalty parameter () is used to weight the importance of classification errors.Meanwhile, the kernel parameter () affects the kernel width.[21,22].DE is a population-based stochastic search engine, which is efficient and effective for global optimization in the continuous domain.It uses mutation, crossover, and selection operators at each generation to move its population toward the global optimum.Superior performance of DE, in terms of accuracy and fast operation, has been verified in many reported research works [21,22].The algorithm (see Figure 1) consists of five main stages: initialization, mutation, crossover, selection, and stopping condition verification.Given that the problem at hand is to minimize a cost function (), where the number of decision variables is , we can describe each stage of DE in detail.

Differential Evolution. This section describes the DE algorithm proposed by Price and Storn
(1) Initialization.DE begins the search process by randomly generating NP number of D-dimensional parameter vectors  , where  = 1, 2, . . ., NP and  represents the current generation.In DE algorithm, NP does not change during the optimization process.Moreover, the initial population (at  = 0) ought to cover the entire search space in a uniform manner.Thus, we can simply generate these individuals as follows: where  ,0 is the decision variable  at the first generation.rand[0, 1] denotes a uniformly distributed random number The diameter through which 10% of the total soil mass passes  10 (m) IF2 The diameter through which 15% of the total soil mass passes  15 (m) IF3 Void ratio  IF4 The fines content of the total soil mass FC (%) IF5 The coefficient of gradation The coefficient of uniformity Water-to-cement ratio of grout w/c between 0 and 1. LB and UB are two vectors of lower bound and upper bound for any decision variable.
(2) Mutation.A vector in the current population (or parent) is called a target vector.Hereafter, the terms "parent" and "target vector" are used interchangeably.For each target vector, a mutant vector is produced via the following equation: where 1, 2, and 3 are three random indexes lying between 1 and NP.These three randomly chosen integers are also selected to be different from the index  of the target vector. denotes the mutation scale factor, which controls the amplification of the differential variation between  2, and  3, . ,+1 represents the newly created mutant vector.
(3) Crossover.The purpose of the crossover stage is to diversify the current population by exchanging components of target vector and mutant vector.In this stage, a new vector, named as trial vector, is created.The trial vector is also called the offspring.The trial vector can be formed as follows: where  ,,+1 is the trial vector. denotes the index of element for any vector.rand  is a uniform random number lying between 0 and 1. Cr is the crossover probability, which needs to be determined by users.rnb() is a randomly chosen index of {1, 2, . . ., NP} which guarantees that at least one parameter from the mutant vector ( ,,+1 ) is copied to the trial vector ( ,,+1 ).
(4) Selection.In this stage, the trial vector is compared to the target vector.If the trial vector can yield a lower objective function value than its parent, then the trial vector replaces the position of the target vector.The selection operator is expressed in the following way: (5) Stopping Criterion Verification.The optimization process terminates if the stopping criterion is met.The type of this condition can be specified by users.Commonly, maximum generation ( max ) can be used as the stopping condition.
When the optimization process terminates, the final optimal solution is readily presented.

Historical Data.
The database used in this research consists of 240 in situ permeation grouting cases [11] for sandy silt soil collected from highway and mass rapid transportation projects.A mixture of microfine cement and microslag in equal proportions was utilized as the injected grout.The diameters through which 95%, 90%, and 85% of the total grout pass are 7.4 m, 6.4 m, and 4.5 m, respectively.Moreover, the diameter through which 70% of the total grout passes is less than 1 m.Thus, the grout is considered to be a seminanometer material.
Recent research works have pointed out that other parameters of the soil and the grout (namely, the fines content of the total soil mass, the water-to-cement ratio of the grout, the void size in soil, the coefficient of uniformity, and the coefficient of gradation of the soil) can be useful for the prediction process [4,[9][10][11].In this study, seven influencing factors are considered to estimate the outcome of a grouting activity (see Table 1).For each data case, the corresponding output is either +1, which means that the grouting is successful, or −1, which indicates unsuccessful grouting.In our research, inherited from the previous research work [11], a grouting process is considered to be successful if the injected grout is at least two times the volume of the void space under the split pressure.Table 2 provides descriptive statistics of influencing factors of the historical data.
Before being used, the data set has been normalized into a [0, 1] range which helps prevent the situation in which inputs with greater magnitudes dominate those with smaller magnitudes [23][24][25].The historical data is illustrated in Table 3.The function used for normalizing data is shown as follows: where   is the normalized data.  is the original data. max and  min denote the maximum and minimum values of the data, respectively.

The Proposed Groutability Estimation Model Based on Autotuning Support Vector Machines (GE-SVM AT )
This section of the paper describes the proposed groutability prediction method, named as GE-SVM AT , in detail.The model (see Figure 2) is constructed by a hybridization of the SVM and DE optimization algorithm.The GE-SVM AT employs SVM as an AI technique for carrying out classification tasks.In addition, the new approach utilizes the DE algorithm for automatically identifying the optimal values of SVM's tuning parameters.The establishment of this prediction model is dependent on two tuning parameters: the penalty parameter () and the kernel function parameter ().Equipped with the DE optimization technique, the SVM algorithm can automatically adapt its parameters according to different learning circumstances without human intervention.
(1) Input Data.The database of 240 in situ grouting cases is divided into training set (90%) and testing set (10%).The training data cases are used to establish the prediction model.Meanwhile, testing data cases are used to verify the predictive performance of the proposed GE-SVM AT .
(2) Tuning Parameter Initialization.The aforementioned tuning parameters of the model are randomly generated within the range of lower and upper boundaries.In this study, the lower and upper boundaries of the tuning parameters are 10 −5 and 10 5 , respectively.Moreover, the equation used for generating the model tuning parameters can be shown as follows: where  ,0 is the tuning parameter  at the first generation.rand[0, 1] denotes a uniformly distributed random number  between 0 and 1. LB and UB are two vectors of lower bound and upper bound for any parameter.
(3) SVM Training Process.In this step, the SVM algorithm is deployed to learn the decision boundary to separate the input data into two classes of groutability (−1 and +1).It is noted that the class label −1 represents a failed grouting process; the class label +1 indicates a successful grouting activity.
(4) DE Searching.The DE optimization approach is applied to explore the various combinations of the tuning parameters ( and ).At each generation, the optimizer carries out the mutation, crossover, and selection processes to guide the population to the optimal solution.By evaluating the fitness of each individual, the algorithm discards inferior combinations of  and  and permits robust combinations of these parameters to be passed on the next generations.
(5) Fitness Evaluation.In GE-SVM AT , in order to determine the optimal set of tuning parameters, the following objective function is used in the step of fitness function evaluation: where AR TR and AR VA denote the classification accuracy rates for the training and validating set, respectively.It is noted that, in the proposed model, the ratio of validating and training cases is set as 1/5.The classification accuracy rate is calculated as the number of correct classification divided by the number of all data instances within a data set.The fitness function represents the trade-off between model generalization and model complexity.It is worth noticing that a good prediction of the training set may reflect the model complexity.However, a complex model tends to suffer from overfitting [26].Overfitting arises when a model predicts the training set very well but performs poorly on the new data set.In order to mitigate the undesirable effect of overfitting, prediction performance of the validating data should be taken into account.Therefore, the proposed fitness function can help identify the model that features the balance of minimizing training error and generalization property.
(6) Stopping Condition.The DE's optimization process terminates when the maximum number of generation ( max ) is achieved.If the stopping condition is not met, the DE algorithm will continue its searching process.When the program terminates, the optimal set of tuning parameters has been successfully identified and GE-SVM AT is ready to predict new input patterns.

Experimental Results
The proposed GE-SVM AT uses 216 data cases for model construction and 24 data cases for testing.This means that 90% of the historical data is used for constructing the prediction model.Meanwhile, 10% of the historical data is reserved for testing process.The groutability results of testing data points are unknown for the model.Therefore, the testing data has the role of new grouting cases which need to be predicted and they can be employed to verify the trained model.
However, due to the randomness in selecting testing cases, the evaluation of model performance can be biased.To avoid such issue, the whole data set (containing 240 cases) is divided into ten data folds in which each fold in turn serves as testing cases, and the model performance can be evaluated via average predictive results of the ten folds.This process is the tenfold cross-validation which is commonly used for verifying model performance [26][27][28].Since all of the subsamples are mutually exclusive, this approach can estimate how accurately GE-SVM AT performs in practice.
Moreover, it is noted that the proposed model utilized the same number of DE generations (G max ) and objective function ( fitness ) for each of the 10 subsets.The parameter  max needs to be sufficient for the DE algorithm to converge; in our research,  max is set experimentally to be 300.Details of the GE-SVM AT prediction results for the first fold are provided in Table 4.Moreover, confusion matrices for all of testing data folds are employed for visualizing the performance of GE-SVM AT (see Table 5).Each column of the matrices represents the instances in a predicted class.Meanwhile, each row indicates the instances in an actual class.Performance of the new groutability estimation method can be evaluated by using the data in the matrices.Observed from the confusion matrices, the proposed GE-SVM AT obtains very high numbers of true positives and negatives.The average true positives and negatives of the proposed method in one fold are 15.1 and 7.3, respectively.Moreover, the numbers of false positives and negatives yielded by GE-SVM AT are remarkably low.The average false positives and negatives in one fold of GE-SVM AT are 0.04 and 0.12, respectively.These results indicate a very robust prediction capability of the new approach.Furthermore, to better demonstrate the capability of the proposed GE-SVM AT , its performance is compared to results acquired from other benchmark approaches including the SVM and ANN algorithms.For these benchmark approaches, similar to the GE-SVM AT , 216 data cases are used for model construction and 24 data cases are reserved for model testing.
When using the ANN algorithm, it is needed to specify the number of hidden layers, the number of neurons in the hidden layer, the learning rate, and the number of training epochs [14].These parameters of ANN are generally selected via repetitive trial and error processes [29].The network configuration is described as follows: the number of hidden layers is set to be 1; the number of neurons in the hidden layer is 7; and the number of training epochs is selected to be 2000.The back-propagation approach [30] is used as the method for training the ANN model.For the SVM algorithm, as suggested by [25], the penalty parameter is 1 and the kernel function parameter is set to be 1/, where  = 7 is the dimension of the input data.
Table 6 provides the result obtained from the tenfold cross-validation of the GE-SVM AT and other benchmark methods.The classification accuracy rates of the GE-SVM AT , ANN, and SVM for training data are 93.0%,92.2%, and 99.6%, respectively.Meanwhile, the classification accuracy rates of the GE-SVM AT , ANN, and SVM for testing data are 93.3%, 89.2.2%, and 73.3%, respectively.
In addition, the average prediction accuracy of AI methods, obtained from the tenfold cross-validation process, and the prediction results of formula-based approaches are provided in Table 7.The main difference between formulabased methods and AI approaches is that formula-based methods only rely on information of the grain size of the soil and the grout.Meanwhile, in addition to that information, AI approaches take into account other influencing factors of the grouting process.Additionally, it is worth noticing that, in Table 7, the capability of groutability prediction approaches is quantified by classification accuracy rates.
Thus, it is obvious that AI methods can deliver much better prediction accuracy compared to formula-based approaches.The experiment has also proven that the integration of SVM and DE can improve the accuracy of the SVM method.Furthermore, the performance of the ANN and SVM methods is inferior to that of GE-SVM AT .GE-SVM AT achieves the best prediction outcome for testing data.Moreover, it can be observed that the proposed model has successfully overcome the issue of overfitting since it yields a relatively balanced performance between training and testing data sets.

Conclusion
This research has presented and verified a new groutability prediction method, named as GE-SVM AT , to assist construction engineers in appraising the possibility of grouting processes that employ microfine cement grouts.The proposed approach was developed by a hybridization of the SVM and DE algorithms.GE-SVM AT utilizes the SVM technique to classify high-dimensional input data so that the model can yield prediction outcomes whenever new input patterns are available.Meanwhile, the DE searching algorithm is implemented to select the most appropriate tuning parameters.Therefore, this mechanism eliminates the need for experience or trial and error process in SVM's parameter setting.Consequently, the newly established model has the ability to operate automatically without human intervention and domain knowledge.Performance comparison has shown the strong potential of GE-SVM AT as an alternative for groutability prediction in the construction industry.The current approach GE-SVM AT is established by historical grouting cases performed in sandy silt soil.Even though experimental results have shown that the proposed method can deliver outstanding prediction results for this type of soil, more historical grouting cases in which the targeted objects of the grouting process involve different soil types should be included to improve the generalization of the GE-SVM AT .Thus, this task can be a promising future direction of this research.

Figure 2 :
Figure 2: Groutability estimation model based on autotuning support vector machine.

Table 1 :
Influencing factors (IF) of historical data.

Table 2 :
Descriptive statistics of historical data.

Table 4 :
The prediction result of GE-SVM AT for the first data fold.

Table 5 :
The result comparison for AI methods using classification accuracy rates (%).

Table 6 :
The result comparison for AI and formula-based approaches.

Table 7 :
GE-SVM AT 's confusion matrices for testing data prediction.