Prediction of Concrete Compressive Strength and Slump by Machine Learning Methods

. Machine learning methods have been successfully applied to many engineering disciplines. Prediction of the concrete compressive strength ( f c ) and slump ( S ) is important in terms of the desirability of concrete and its sustainability. The goals of this study were (i) to determine the most successful normalization technique for the datasets, (ii) to select the prime regression method to predict the f c and S outputs, (iii) to obtain the best subset with the ReliefF feature selection method, and (iv) to compare the regression results for the original and selected subsets. Experimental results demonstrate that the decimal scaling and min-max normalization techniques are the most successful methods for predicting the compressive strength and slump outputs, respectively. According to the evaluation metrics, such as the correlation coeﬃcient, root mean squared error, and mean absolute error, the fuzzy logic method makes better predictions than any other regression method. Moreover, when the input variable was reduced from seven to four by the ReliefF feature selection method, the predicted accuracy was within the acceptable error rate.


Introduction
Concrete is a complex composite material. e predictability of concrete properties is extremely low. erefore, it is challenging to model the concrete properties according to the effect variables. e biggest challenge of experimental designs is a high number of effect variables affecting the response variables. Multiple effect variables increase the number of trials. e higher amount of uncontrollable variables makes it difficult to obtain the real response function.
Generally, the one-factor-at-a-time method is used in experimental designs to determine the concrete properties. e major disadvantage of this approach is that it does not consider the interaction between the factors (interaction terms). e higher the number of the controlled and uncontrolled effect variables that influence the concrete properties, the lesser the predicted accuracy. Despite this, a few experimental designs have been suggested by considering the controllable effect variables and interaction terms between them [1].
Machine learning (ML) is a highly multidisciplinary field and consists of various methods for obtaining new information [2]. ML is most often used for prediction. Predicting the categorical variable values is called classification, whereas predicting the numerical variable values is called regression. Regression is the process of analyzing the relationship between one or more independent variables and a dependent variable [3].
In recent years, the ML methods have become popular as they allow researchers to improve the prediction accuracy of concrete properties [4] and are used for various engineering applications [5,6]. e ML methods have been used to increase the prediction accuracy of concrete properties [7][8][9][10][11][12][13][14][15], and the data derived from the literature sources were used. However, Chopra et al. [16,17] applied the data generated under the controlled laboratory conditions.
Regression models tend to be used for the prediction of the compressive strength of high-strength concrete [18,19]. ese models also demonstrate how the concrete compressive strength depends on the mixing ratios [20]. Topçu and Sarıdemir [21] and Başyigit et al. [22] developed models using the neural network (NN) and fuzzy logic (FL) methods to improve the prediction accuracy of the compressive strength of the mineral-additive (fly ash) concrete and heavy-weight concrete. Both studies concluded that the compressive strength could be predicted by using the models that were developed with the NN and FL methods without any further experiments. NN is more successful than the data mining methods and does not enhance the prediction accuracy of the concrete compressive strength [15,17,[23][24][25][26]. Khademi et al. [27] compared the multiple linear regression, neural network, and adaptive neuro-fuzzy inference system (ANFIS) methods to estimate the concrete compressive strength for 28 days and reported that the NN and ANFIS models provide reliable results.
Previous studies evaluated the amount of the concrete component materials and compared their results to the published data. In this study, the ML regression methods were compared to predict the compressive strength and slump values of the cube samples. e samples were prepared by accounting for seven simultaneously controllable effect variables in the laboratory. e study aimed to determine the most successful regression method by comparing the decision tree (DT), random forest (RF), support vector machine (SVM), partial least squares (PLS), artificial neural networks (ANN), bootstrap aggregation (bagging), and FL models for the prediction of the concrete compressive strength and slump values. e R, RMSE, and MAE metrics were used to compare the prediction accuracy of the developed models. Finally, feature reduction was accomplished by the feature selection method. en, the model's success rates were compared to predict the compressive strength and slump value using fewer variables.

Experimental Datasets.
Datasets used for this study comprised seven input variables (i.e., W/C, C, f cc, FA, k k , CA, and TA) and two output (response) variables (i.e., f c and S) for two different maximum aggregate sizes D max. � 22.4 mm (D 224 ) and D max. � 11.2 mm (D 112 ). e input variables were selected considering the simultaneously controllable effect variables [28][29][30]. D-optimal design obtained by the augmentation of the fractional factorial design (2 7-3 ) was used as the experimental design. In the D-optimal design, 58 and 56 test results were employed for D 112 and D 224 , respectively. Each experimental result was calculated as an average of three sample results that are produced under laboratory conditions [28][29][30]. Properties of the constituents are given in Table 1 [28][29][30]. Abbreviations of the effect and response variables and the basic statistic of the datasets are presented in Table 2.

Methods
In this study, the concrete compressive strength and slump values were predicted using the ML regression models, namely, the regression tree, RF, support vector machines, artificial neural network, partial least square, bagging, and FL. Datasets were randomly split into 70% for the training set and 30% for the independent test set. e training data were used to train the ML model. e independent test data were applied for the evaluation of the model's performance. e 10-fold cross-validation procedure helped in the estimation of the ML model skills.
e ML preprocessing steps were applied to the raw datasets before they could be utilized for the regression method training. e datasets were not normally distributed according to the Shapiro-Wilk normality test [31] results. Many normalization methods have been previously developed to normalize the dataset [32]. In this study, four different normalization methods (i.e., minmax, decimal, sigmoid, and z-score) were applied to derive the most successful normalization method for the raw dataset. en, the K-nearest neighbor (KNN) regression method was applied to the normalized datasets. e prediction results were compared to determine the most suitable normalization method. Later, the raw datasets were normalized with the determined normalization technique.
e ML regression models were trained to predict the f c and S values. e correlation coefficient (R), root mean squared error (RMSE), and mean absolute error (MAE) metrics were employed to compare the models' prediction performance. According to these statistical results, the most successful regression method was determined to predict the f c and S values. Afterward, the feature selection method was used to obtain the subset with fewer features, and the prediction accuracy was examined. All regression methods and computations were performed using the R programming language [33]. e prediction process is illustrated in Figure 1 in the form of a flow diagram.

Normalization Methods.
Normalization is the preprocessing step in ML. e normalization methods are used where the variation intervals of the variables in the dataset differ. When the mean and variance of the variables differ significantly, the variables with a large mean and variance increase the impact on the other variables. is may result in the loss of important variables due to the low variation intervals. It can also affect the success of the ML models [34,35]. erefore, regression models are normalized by the numerical data normalization methods to standardize the effect of each variable on the results. In this study, the dataset was normalized by the min-max, decimal, sigmoid, and z-score normalization techniques, and then their performances were compared.

Machine Learning Methods.
e ML regression method estimates the output value using the input samples of the dataset. Such a procedure is also termed as the training set. e purpose of the regression method is to minimize the error between the predicted and actual outputs [36]. Herein, seven different regression methods (i.e., DT, RF, support vector machine, partial least squares, artificial neural networks, bootstrap aggregation (bagging), and FL) were used to predict the concrete compressive strength and slump values. Additionally, the K-nearest neighbor method was applied to determine the suitable normalization method for the dataset. ese methods are briefly described below.
Decision tree (DT) [37] is a supervised ML algorithm. It can be used for both regression and classification. e aim of the DT algorithm is to divide the dataset into smaller, meaningful pieces, where each input has its own class label (tag) or value. Different measurements are used for the DT splitting, such as Gini and information gain. Regression tree is a type of a DT and a hierarchical model for the supervised learning. Classification and regression trees (CART), ID3, and C4.5 methods are the most important learning algorithms mentioned in the literature. In this study, the CART [38] model is used for the regression.
Random forest (RF) [39] is an ensemble method that combines many DTs. It can be used for both regression and classification. Each DT in the forest is created by the selection of different samples from the original dataset by the bootstrap technique. ese samples are then trained using a set of attributes selected by the bagging mechanism. Subsequently, the decisions made by a large number of individual trees are subjected to voting. As such, the most voted class is presented as the class estimate of the community. Support vector machine (SVM) has been developed by Vapnik [40]. It is applied both for regression and classification. e SVM method is based on finding an optimal hyperplane that maximizes the margin between the classes.

Advances in Civil Engineering
Partial least squares (PLS) [41] regression generalizes and combines the attributes from the principal component analysis and multiple regression. e most important characteristic of the PLS method is its ability to obtain a simple model with a few components, even when the variables are highly correlated or linearly independent.
Artificial neural networks (ANN) [42] involve a system of many interconnected neurons. e neurons are connected  by the weighted links. e ANN architecture consists of the input, hidden, and output layers. e multilayer perceptron neural network (MLP) is a fully connected, feedforward type of network. It is mostly used in network architecture. e output of all the neurons in the input layer is scaled by the related connection weights. en, the input of the neurons is feedforwarded to the output layer. Activation functions are used for the sum of the input neuron signals in the output layer.
Bootstrap aggregation (bagging) was introduced by Breiman [43] and can be utilized for both regression and classification. Bagging is performed by aggregating the resulting prediction rules using the bootstrap samples from the training sample.
Fuzzy logic (FL) is an ML method and was introduced by Zadeh [44]. FL is a mathematical-based method used to analyze the systems in a manner similar to how people do. As many problems could not be expressed by the exact mathematical definitions, a new method was developed. In the classical approach, an element is a member or nonmember of the cluster, making the result equal to zero or one. However, in the FL, the situation is expressed by the membership degrees, which indicate the element's involvement in the cluster. e membership function is used to map each element into a continuous interval from zero to one. In other words, the membership degree of the element can vary as an infinite number from zero to one. A typical fuzzy system consists of a rule base, membership functions, and inference procedure. In this study, Wang and Mendel's technique (WM) was employed to generate the fuzzy rule.
K-nearest neighbor (KNN) [45] is an instance-based algorithm and can be applied for both regression and classification.
e KNN method searches for the k-data points closest to the test object and uses the features of these neighbors to classify the new object. For this, a distance is measured between each instance in the training dataset and the test instance. Herein, k � 3, 5, and 7 were chosen. e Euclidean distance was deployed as a distance measure. e "knn.reg" function was used in the "FNN" package [46]. e detailed information regarding the ML regression methods applied in this study is presented in Table 3.

Evaluation Metrics.
To evaluate the predicted values of the regression methods, the actual and predicted values were compared. In this study, the R, RMSE, and MAE metrics were used to evaluate the prediction accuracy [47]. e model parameters were optimized for the highest R, lowest RMSE, and lowest MAE. All of them were calculated according to the following equations: Here, N is the number of data points.

Feature Selection.
Feature selection (reduction in irrelevant variables) is the preprocessing step in ML that selects the best subset from the original dataset by evaluating the properties according to the used algorithm [48]. e ReliefF algorithm was developed by Kira and Rendell [49]. It weights the features according to the relationship between the effect variables. Although this method was successfully applied to two classes of the datasets, it was not proved functional for the datasets with multiple classes. To solve this problem, in 1994, Kononenko developed the ReliefF algorithm that works for the multiclass datasets [50]. e algorithm determines the weights of the continuous and discrete attributes based on a distance between the instances.

Results and Discussion
e cross-correlation between the datasets representing the parameters D 112 and D 224 is depicted in Figure 2. e correlation coefficient provides information on the effect level and direction of the linear relationship between two variables. e Pearson correlation is used when the dataset has a normal distribution, whereas the Spearman correlation is applied when the normal distribution cannot be reached.
According to the correlation results of the D 112 dataset, the response variable f c is highly correlated with the effect variable f cc (0.88). Moreover, the highest correlation is observed between the response variable S and the effect variable TA (− 0.57 for basalt and 0.57 for limestone). According to the correlation results of the D 224 dataset, the response variable f c is highly correlated with the effect variable f cc (0.91). Besides, the highest correlation is obtained between the response variable S and the effect variable TA (− 0.55 for basalt and 0.55 for limestone).
Before the data analysis begins, the data must be checked in accordance with the normal distribution. In this study, the normality test was performed using the Shapiro-Wilk normality test with the Gaussian error [51]. In the Shapiro-Wilk normality test, when the probability is >0.05, the data are normally distributed, whereas when the probability is <0.05, the data demonstrate a nonnormal data distribution. e small W value in the Shapiro-Wilk normality test indicates that the sample is not normally distributed.
e Shapiro-Wilk normality test results for the D 112 and D 224 datasets are presented in Table 4.
According to the Shapiro-Wilk normality test results (Table 4), the D 112 and D 224 datasets are not normally distributed with the probability of the variables <0.05 and very high W values for both datasets. Furthermore, the boxplot graphs (Figure 3) prove that the dataset is not normally distributed. With the box-plot graph, it is possible to examine both ranges of the value and the numeric variable distribution.

Advances in Civil Engineering
In this study, four different normalization techniques, namely, min-max, decimal, sigmoid, and z-score were applied to four different datasets. As a result, the most successful method was determined. After the normalization of the datasets by these methods, their success rate was compared using the KNN regression method. e KNN regression method was chosen being distance-based and rapid in application. In this study, the k-values were selected at 3, 5, and 7. e results are provided in Table 5 Table 6. e reason for the selection of these regression methods was the successful employment of those prediction algorithms in published literature. As mentioned earlier, the datasets were randomly divided into the training (70%) and individual test sets (30%). Herein, the training and individual test sets consisted of 40 and 18 instances for the D 112 dataset, respectively, and 39 and 17 instances for the D 224 dataset, respectively. Prediction results for the models were obtained from the 10-fold cross-validation process. e    Advances in Civil Engineering performance of these regression methods was evaluated according to the R, RMSE, and MAE statistical criteria. R was employed to evaluate the good fit between the predicted and actual values. A combination of the R, RMSE, and MAE results was sufficient to reveal any significant differences between the predicted and actual values. According to the statistical results of the regression method (Table 6), the FL regression model delivered the   To reduce the number of the effect variables, the ReliefF feature selection method was used to determine the highlevel effect variables. As a result, the f cc , k k , C, and W/C effect variables were selected as they had a high-level effect on f c and S for the maximum aggregate size. e results of R, RMSE, and MAE obtained after applying the FL model to all the effect variables and reduced effect variables are presented in Table 7. is table also indicates that there is no significant change in the R, RMSE, and MAE results when the number of features was reduced from seven to four. erefore, the FL model with fewer features can still make successful predictions. e effect levels of the simultaneously controllable effect variables on the response variables exhibit some variations [28][29][30]. Considering the selected variation intervals, the f cc , k k , C, and W/C variables had a significant effect level on the response variables for the maximum aggregate sizes. Cement strength (f cc ), cement dosage (C), and water/cement (W/C) ratio tend to have a significant effect on the compressive strength. Furthermore, the fineness modulus (k k ), which expresses the fineness and distribution of the mixture aggregate, is one of the essential variables that affects the concrete compactness. Moreover, the concrete compactness directly affects the compressive strength.
e workability of concrete is directly influenced by the cement properties (e.g., cement fineness), aggregate properties (e.g., roughness of the aggregate surface), and amount of mixing water. Particularly, it is not expected that the chemical additive variable does not have a significant effect on workability. e variation intervals of the chemical additive are negligible and do not show a significant effect on the workability of concrete. However, the variation intervals of the other effect variables can be considerable. erefore,

Conclusions
e goals of this study were (i) to determine the most successful normalization technique for the datasets, (ii) to obtain the prime regression method to predict the f c and S values, (iii) to choose the best subset using the ReliefF feature selection method, and (iv) to compare the regression results for the original and selected subsets.
To determine the effect levels of the effect variables on the response variables (i.e., f c and S) with precision, data were analyzed for normalization. If the data were not normally distributed, it was necessary to determine the most appropriate normalization method. In this study, the Shapiro-Wilk normality test results demonstrated that the datasets were not normally distributed. e most successful techniques for the determination of the f c (D 112 , D 224 ) and S (D 112 , D 224 ) values were the decimal scaling and min-max normalization methods, respectively. erefore, as the variation ranges of the effect variables influencing the concrete properties varied substantially, it was necessary to preprocess the raw data for the estimation of the concrete properties.
Herein, seven different ML methods, such as DT, RF, SVM, PLS, ANN, bagging, and FL were experimented with to predict the f c and S values. According to the R, RMSE, and MAE statistical results, FL is the best regression method for the maximum aggregate size. Generally, the similarity between the actual and predicted values is high for the compressive strength (Figures 4 and 5). A minimal difference between the actual and predicted slump values indicates that the slump values are more sensitive to the experimental error, simultaneously uncontrollable effect variables, and variation intervals of the effect variables. e flexibility of the computational structure of the FL approximated the results instead of providing the exact results. In particular, the uncertainties in the problem-solving and decision-making processes can be clarified by the application of the FL. us, complicated problems can be solved, making the FL more functional than any other ML method.
In experimental designs, where the number of the simultaneously uncontrollable effect variables is high, it is crucial to reduce the number of the experiments to save costs and time. erefore, the predicted values close to the actual values need to be obtained with the minimum number of the experiments. In this study, seven simultaneously controllable effect variables were reduced to four effect variables (i.e., f cc , k k , C, and W/C) using the RelifF feature selection method. e metric results obtained by the FL regression were similar for four and seven effect variables (Table 7). erefore, the experimental designs with fewer effect variables are sufficient for estimating the concrete properties.

Data Availability
Previously reported "compressive strength and slump of concrete" data were used to support this study and are available in the author's PhD thesis, report, and article. ese prior studies (and datasets) are cited at relevant places within the text as references [28][29][30].

Conflicts of Interest
e author declares that there are no conflicts of interest. Advances in Civil Engineering 9