Compressive Strength Prediction Using Coupled Deep Learning Model with Extreme Gradient Boosting Algorithm: Environmentally Friendly Concrete Incorporating Recycled Aggregate

The application of recycled aggregate as a sustainable material in construction projects is considered a promising approach to decrease the carbon footprint of concrete structures. Prediction of compressive strength (CS) of environmentally friendly (EF) concrete containing recycled aggregate is important for understanding sustainable structures’ concrete behaviour. In this research, the capability of the deep learning neural network (DLNN) approach is examined on the simulation of CS of EF concrete. The developed approach is compared to the well-known artiﬁcial intelligence (AI) approaches named multivariate adaptive regression spline (MARS), extreme learning machines (ELMs), and random forests (RFs). The dataset was divided into three scenarios 70%- 30%, 80%-20%, and 90%-10% for training/testing to explore the impact of data division percentage on the capacity of the developed AI model. Extreme gradient boosting (XGBoost) was integrated with the developed AI models to select the inﬂuencing variables on the CS prediction. Several statistical measures and graphical methods were generated to evaluate the eﬃciency of the presented models. In this regard, the results conﬁrmed that the DLNN model attained the highest value of prediction performance with minimal root mean squared error (RMSE � 2.23). The study revealed that the highest prediction performance could be attained by increasing the number of variables in the prediction problem and using 90%-10% data division. The results demonstrated the robustness of the DLNN model over the other AI models in handling the complex behaviour of concrete. Due to the high accuracy of the DLNN model, the developed method can be used as a practical approach for future use of CS prediction of EF concrete.


Introduction
Consideration of sustainable development is an important requirement in the new era of the construction industry [1,2]. Sustainable development is applied by reducing environmental impact and protecting natural resources. Increasing population growth led to the high demand for construction and, as a result, depletion of natural resources needed in concrete construction [3,4]. Recently, many researchers have studied the production of environmental friendly (EF) concrete, which has less harmful effects on nature [5][6][7][8]. One of the methods studied by the researchers is using recycled aggregate [9]. Production of construction debris and waste has been increasing using recycled materials in concrete production in the last years. e use of recycled aggregate in a construction project is considered a promising method to decrease the carbon footprint of a concrete project [10][11][12][13]. In recent years, EF concrete has been studied experimentally and mathematically [14,15].
Early investigation on the mechanical behaviour of EF concrete was conducted using stone dust in concrete production [16]. e modelling results showed that the compressive strength of EF concrete is higher. e durability of concrete was studied and confirmed to produce high performance using recycled material [14]. Two researches have adopted an investigation of experimental analysis for the effect of using additive material such as fly ash and silica fume beside recycled aggregate on the mechanical properties of concrete [17,18]. e studies indicated that using additive materials and recycled aggregate can enhance the strength and durability of concrete. Recycled aggregate usage in concrete enhances pore distribution and frost resistance [19]. e addition of agriculture waste to the concrete mix was considered and was investigated to enhance the properties of concrete using silica fume [20]. Calcium carbide residue was examined over the literature as industrial material [21]. Compressive strength and failure modes of concrete containing recycled nylon fiber fabric were studied [22]. Other researchers confirmed the potential of the produced EF concrete, and some researchers studied the utilization of agriculture and local waste in Africa for concrete production [23]. Others discussed the properties of lightweight interlocking concrete by adding sawdust and laterite in the concrete production [24]. All the reported literature on the experimental developed EF concrete elaborated the significance of producing such product material that can help essentially on the construction field and material sustainability. In addition, studies showed the importance of using recycled waste as environmental material in concrete production.
Owing to the complexity and nonlinearity of concrete behaviour and various parameters that affect CS, the necessity of developing an effective technique that can examine the properties of concrete can be noted. In recent years, artificial intelligence (AI) models have been effectively applied in concrete structure research [25][26][27][28][29]. AI models have the ability to solve nonlinear, stochastic problems and deal with complex systems [30][31][32]. Literature studies on AI models reported massively in this domain; artificial neural network (ANN) model with response surface methodology (RSM) was developed for predicting CS of recycled aggregate concrete [33]. e study concluded that the ANN-based model exhibits better performance than the RSM model. e development of adaptive neuro-fuzzy inference system, ANFIS, together with ANN model was adopted to estimate self-compacting concrete's CS [34].
e results demonstrated that the developed model accurately predicted concrete compressive strength. In another study, adaptive boosting (AdaBoost), ANN and support vector machine (SVM) were developed for the prediction process of compressive strength [35]. e study demonstrated that Ada-Boost model exhibited better prediction achievement than other models. Examination of the feasibility of ANN model in the prediction process of concrete CS involving furnace slag of ground granulated blast was conducted by [36]. e study was integrated ANN with a multiobjective salp optimization algorithm and compared it with M5P model for CS prediction. e research was concluded that the proposed AI models successfully predicted CS. e incorporation of socio-political algorithm (ICA) with extreme gradient boosting (XGBoost), ANN, SVM, and ANFIS was tested for predicting of CS of recycled aggregate concrete, and was reported by [26]. e study revealed that the developed model (ICA-XGBoost) achieved better capacity than the other proposed models in the prediction process.
e ability of ANN model was evaluated in the prediction of CS of Geopolymer Concrete (GPC) [37]. e results indicated that the proposed model attained high prediction capability of compressive strength. Boosted decision tree regression (BDTR) and SVM models were developed to predict EF concrete's compressive strength [38]. e researcher concluded that the BDTR model performed better than the SVM model with good accuracy prediction. e modelling accuracy of ANN, RSM, and gene expression programming (GEP) was tested for predicting the CS of engineering Geopolymer composites concrete [39]. e study showed that ANN and RSM models performed better than GEP model in the prediction process. Hybridization of three optimization algorithms called genetic algorithm (GA), salp swarm optimization (SSA), and grasshopper optimization algorithm (GOA) with ANN model were conducted for predicting CS of the concrete utilizing recycled aggregate [40]. e researchers indicated that ANN-SSA achieved better prediction performance than other models. Several other researches were adopted on testing the feasibility of the AI models for modelling the CS of concrete and confirmed the potential of those new computer-aided models in solving such a kind of complex material engineering problem [41][42][43][44][45].
Previous studies show that the use of recycled aggregate as a sustainable material in concrete production is highly emphasized by several researchers [46,47]. Most of these studies examined the impact of recycled aggregate experimentally and there is a limited number of researchers who have explored the properties of EF concrete by a mathematical model. Due to the rapid advancement of AI models and computer vision, exploring a new approach that can handle the complex relationship between different parameters and the compressive strength of concrete is very important. Deep learning neural network (DLNN) is a new version of AI models that have achieved a reliable performance in solving complex and nonlinear systems [48][49][50]. DLNN outperformed other traditional AI models in various applications owing to their capacity on analyzing through the deep layers learning process. DLNN has been successfully applied and performed better prediction precision in several engineering problems [51][52][53][54]. e other concern that highly matters to the machine learning modelling is the feature selection for the appropriate 2 Complexity predictors. Hence, the current research focused on integrating a new feature selection approach. is paper aims to provide an effective computer-aided model for predicting CS of recycled aggregate concrete.
is approach can help determine the performance of EF concrete on the improvement of concrete strength and identify the appropriate mix designs of concrete. To achieve the research aim, the DLNN model is developed and compared with extreme learning machine (ELM), multivariate adaptive regression spline (MARS), and random forest (RF). Due to the important role of input parameters in CS prediction, XGBoost as an advanced algorithm is used that abstracts the most correlated variables in the modelling process. e proposed approach is helpful in many applications of a concrete structure and can be applied to examine the compressive strength of EF concrete.

Deep Learning Neural Network (DLNN).
In recent years, the concept of the deep learning method has been introduced as an advanced algorithm of neural networks. Deep neural network is a traditional neural network with extra numbers of hidden layers that are added to the structure [55]. e deep learning algorithm was developed by Hinton et al. as he introduced a layer-wise-greedy-learning method [56]. According to this method, an unsupervised learning method is used to pertain the neural network before the training phase with layer by layer. Herein, deep learning is a popular AI model for some reasons: its ability to deal with big training data, avoid overfitting problems, and nonrandom value can be assigned to the network before learning process [55]. From these features, the algorithm can make a good performance through the training process among deep learning types. e backpropagation neural network is used in this study due to its popularity and robustness in many applications [51].
is type used multiple hidden layers and backpropagation with gradient descent algorithm. e main concept of this algorithm is yielding new variables based on the connection between hidden layers and the input layers. e new variables are reached to the output layer, and then the target value is predicted through the training process. e nonlinear relationship between multiple hidden layers gave the algorithm the capability to handle nonlinear relationships in different complex systems [57]. e structure of a deep neural network is shown in Figure 1 with the input layer, hidden layers, and output layer. e mathematical expression of the algorithm is discussed as follows: where f is the activation function, b is the bias, and w is the weight matrix of the hidden neurons. Input layer is described by 0 and L represents the output layer. e activation matrix used in this study is tanh function because of its capability to get good prediction performance in the explored problem.

Multivariate Adaptive Regression Spline Model (MARS).
Multivariate adaptive regression spline model, a robustn machine learning model, was developed by [58]. MARS is a nonlinear flexible model recently used in many applications in engineering subjects [59]. e MARS model has three principle components: the basic functions (BFs), spline functions, and the knots [60]. e relationship between target value and predicted variables is addressed by BFs and it is expressed by max (0, c − x) or max (0, x − c), where x represents threshold value and c indicates the value of input variable. In this context, the knots indicate the function for the base as well as base endpoints. Spline functions contain one or more BFs and its role is developing a regression model for each node [61]. e predicted value in MARS model is based on linear combination of BFs components. e main processes of MARS model are as follows: Assuming Y represents the target variable whereas X is the matrix of input variable, then the mathematical expression of MARS model is described as follows: where β 0 represents the initial value; and BF m is the basis function that is used to fit MARS model; whereas M denotes the whole number of BFs [62]. ere are two main phases in MARS model named forward phase and backward phase as shown in Figure 2. In forward phase, the model selects the optimum combination of input parameters. e overfitting problem can appear in the forward phase due to a series of splits, and the model cannot achieve good performance during the prediction process. e model uses the backward phase to remove undesirable parameters that have been selected and then enhance the prediction accuracy of a regression model. e basis deletion criteria in the backward phase are a generalized cross validation, which is indicated as GCV and can be computed as follows: where O i denotes the observed value; N is the total number of data; f(x i ) is the predicted value of x, M represents the whole number of BFs; and (C(M)) indicates the penalty factor. d represents the optimization cost of BFs and its range 2 ≤ d ≤ 4. e systematic structure of MARS model is described in Figure 2.

Extreme Learning Machine (ELM)
. ELM is one of the advanced models that have been used in recent years to train the developed single layer feedforward neural network [63]. Each layer in ELM model has a number of neurons and the model contains input, hidden, and output layer. e sigmoid function is commonly utilized as activation function for the hidden layer, while the activation function in input and output layer is linear Complexity function [64]. To achieve the optimum linear system solution, developing the ELM model required the determination of random input weight and hidden biases, then using the Moore Penrose generalized inverse method [65,66]. ELM model has been used in many engineering tasks owing to fast learning, no parameter tuning, and its strong ability to achieve generalized model [67][68][69]. In ELM model, the first step is linear mapping the input vectors into an L-dimensional feature map as shown below: Output summation of the BF Over-fitted input data Piece wise linear/piece wise cubic equation

Input parameters
where N denotes the size of training data, t i represents the output vector, and x i the input vector. β i refers to the weight vector which represents the connection between hidden neuron and output layer; w l is the weight vector that links the hidden neuron to the first layer; b l represents the bias; while g symbolizes the activation function. In this regard, the structure of ELM model is illustrated in Figure 3.
According to the ELM model, a traditional single layer ANN can handle all sample data zero variation as shown in the below mathematical expression: where t i represents the output vector. x i denotes the input vector. Herein, the previous equation can be reconstructed as shown below: where where β indicates the weight for the matrix which links hidden and output layer; H is output matrix of hidden layer; and T is the output matrix of target value based on N training data. By assuming that hidden biases along with input weight are both constant; it suggests that the model can be described as linear system where H and T indicate an output matrix from hidden layer and output target. While β is the weight of the matrix optimized during the training phase. e least-squares solution of the linear system can be expressed as follows: where H † represents Moore Penrose generalized inverse matrix.

Random Forest (RF).
Random forest (RF) is a robust AI method based on classification and regression tree and developed by [70]. RF model was applied in classification and regression problems and gave an excellent performance in many fields of engineering applications [71,72]. e work of RF model is based on the abstraction of multiple samples from the original sample by using the bootstrap resampling method, and then a decision tree model is developed for each bootstrap sample, and various outputs of decision tree models are combined to get the final result of prediction.
Modifying the model parameters led to the use of many types of decision trees and then developed different trees models. e modelling process of RF algorithm includes the following phases: using bootstrap resampling method to extract N predictors training data from the original problem data [73]. During training phase, the training data of RF model should be equal to 2/3 of the original data size. e utilized data is called in-bag data and the remaining 1/3 dataset is called out-of-bag data. e regression tree (RT) is developed for each training bootstrap data and a random forest algorithm is built based on the developing N predictors of RT models [74]. e variation of regression trees can be determined during the training data process by randomly selecting the optimal attribute from the maximum depth attributes; this process increases the capability of the RF model. e sequence of regression model can be attained by training the algorithm several times and this sequence is helpful for the developing of random forest system. e prediction results of N predictors are collected and final regression model for the new sample is calculated by using the average method [75]. e mathematical equation of regressions model in RF algorithm is as follows: where f K rf (x) represents the regression model of random forest, t i is the regression model of each regression tree, and K represents the number of regression tree model. e structure of RF model is shown in Figure 4:

Extreme Gradient Boosting (XGBoost).
XGBoost is an advanced algorithm that was used recently in the feature selection process. It was developed as an extension and improvement of gradient boosting tree [76,77]. e main structure of the algorithm depends on the efficient construction of boosted trees [78]. In feature selection problem, the main objective of XGBoost is to construct boosted trees to achieve the feature importance of input variables that are used for training process [79]. Boost trees are classified into classification and regression trees. e features' importance is extracted using three methods: gain, frequency, and cover [80]. e gain method calculates the importance of the features, frequency computes the number of trees in the boosted trees, and cover calculates the relative value for the observation [79]. e feature importance is calculated by weight using XGBoost as follows [81]: where L represents the number of trees, N indicates the number of leaf nodes, (V l L ) is the feature of node l, and I is the indicator function. e statistical characteristics of the datasets are reported in Table 1. ese statistical characteristics include maximum value, minimum value, mean, standard deviation, skewness, and kurtosis. Figure 5 demonstrates the histogram for all input and output variables. e CS of the EF concrete ranges from 13 to 88.3, with mean value equal to 42.11, indicating high variance and giving the noticeable complexity of prediction problem. Table 1 and Figure 5 show that the data are well distributed and nearly to normal distribution. e correlation matrix between input and output variables is depicted in Figure 6. Correlation statistics showed a good correlation between cement quantity and compressive strength. Increased cement quantity led to increasing the compressive strength of concrete. Figure 6 also demonstrates that there is a small correlation between value of coarse aggregate and compressive strength. ere is also a small negative correlation between water, fine aggregate, recycled aggregate, and compressive strength. Increases of water, fine

Hidden layers
Output layer Four AI models were adopted for the modelling process, i.e., DLNN, ELM, MARS, and RF. Dataset was split into three scenarios including 70-30%, 80-20%, and 90-10% to examine the best size of training and testing dataset suitable for modelling problems. XGBoost was used as a robust model to abstract the highly correlated variables of the best size of training and testing dataset for CS prediction. e input combinations of CS prediction are reported as in Table 2. It can be noted that the most relevant variable of CS prediction is the quantity of cement. e second input combinations of CS prediction are quantity of cement and water. Input combinations of XGBoost model showed that negatively correlated parameters and cement quantity are important variables in CS prediction. e procedure of the developed AI model is illustrated in Figure 7. e accuracy of modelling is evaluated by using different statistical indicators including determination coefficient (R 2 ), mean absolute percentage error (MAPE), root mean square error (RMSE), mean absolute error (MAE), Nash-Sutcliffe efficiency (NSE), and Willmott' index (WI) [83,84].    8 Complexity where y p and y o represent the predicted and observed value of compressive strength; y o is the mean value of the observed value of compressive strength; and N represents the number of experiment samples.

Results and Discussion
In this research, four AI models were established to predict the compressive strength of EF concrete containing recycled aggregate. e main idea from developing AI models is to examine the ability of the developed model to predict the compressive strength of EF concrete, and therefore three data division scenarios were used to test the size of dataset suitable for CS prediction. Tables 2-4 describe the statistical indicators of the developed models (DLNN, MARS, ELM, and RF) over the training phase for the three data division scenarios. In general, the results indicated that all the developed models performed good prediction performance. is indicated that the developed models are reliable and they are capable to build a robust approach of CS prediction. However, the reported results showed the superiority of DLNN model over the other developed models through all data division scenarios. e prediction performance is gained by using DLNN model for 80 (Tables 6-8).
Scatter plots were created to examine the linear relationship among the actual and predicted values for EF concrete compressive strength. e scatter plots for the three scenarios (70-30%, 80-20%, and 90-10%), and the developed AI models (DLNN, MARS, ELM, and RF) are depicted in Figures 8-10. e 90-10% scenario achieved the best linear relationship between actual and predicted compressive strength values for the testing period using the DLNN model. Figure 10 indicates that the best prediction improvement is attained by the DLNN model with a maximum coefficient of determination R 2 � 0.97.
A two-dimensional graphical presentation named Taylor diagram is generated for investigating the performance for the developed AI model [85]. Taylor diagram is developed based on three statistical metrics correlation, RMSE, and standard deviation of the developed AI models as depicted in Figure 11. According to the location of the developed AI models on Taylor diagram, it can be noted that DLNN model achieved the best prediction performance for compressive strength of EF concrete by using a data division scenario of 90-10%.
Another graphical presentation is created to give a good explanation for the performance of the developed AI models. e developed AI models were investigated by using relative error percentages for the three tested data division scenarios. Figure 12 illustrates the relative error results of the developed model for scenarios 1, 2, and 3. According to the relative error results, the DLNN model attained the best predictive performance with a small relative error for scenario 3 with 90-10% data division.
Tables 9-12 present the statistical measurement for integrating XGBoost combinations with the developed AI models over training and testing phase for scenario 3. e results indicated that DLNN and RF models performed better prediction than ELM and MARS models. e results also reported that the best results were achieved by including al input parameters in the modelling problem. e best prediction accuracy during training phase was attained by DLNN Figure 13 demonstrates the scatter plot diagram for all the developed model combinations. e DLNN model achieves the best results with highest value of the coefficient of determination R 2 � 0.97. e diagram shows that the best prediction performance is achieved by using all influencing input variables. Applying all input variables increases prediction performance for all developed models (DLNN, MARS, ELM, RF). e Taylor diagram demonstrates the relationship between correlation, RMSE, and standard deviation for all       input combinations, as shown in Figure 14. Taylor's presentation revealed that the developed AI models achieved good prediction using sixth combinations with all input parameters. DLNN model attained the best prediction performance using all input variables with the nearest position to the actual value.
Relative error presentation for the proposed AI models with all abstracted combinations is illustrated in Figure 15. Graphical presentation of relative error percentages showed that modelling with six parameters attained low percentage error compared with the other combinations for all AI models. DLNN model with the sixth combination achieved the lowest relative error among the other three AI models (MARS, ELM, and RF).
In the current study, a robust AI model was suggested for predicting the compressive strength for EF concrete. An advanced XGBoost algorithm was integrated with the developed AI model and the comparable once to investigate the best combinations of input parameters suitable for predicting EF concrete compressive strength. e results reported that the presented models realized an efficient prediction model that has the capability to handle the complex behaviour of concrete.
is approach can improve concrete performance and the strength of EF concrete. Based on the reported results, it was observed that the size of data used for training AI model remarkably affects the accuracy of prediction performance. An accurate selection of training data size helps the modeller reduce an underfitting problem associated with the modelling process. e current study presented that 90% of training data is suitable to provide an accurate prediction of EF concrete compressive strength.
e integration of XGBoost algorithm with AI models showed that an accurate prediction could be achieved by including all input parameters in the prediction problems. All six variables including cement, water, fine aggregate, recycled aggregate, age strength, and coarse aggregate are necessary to achieve good prediction accuracy of compressive strength.
e results also reported that DLNN model provided an excellent ability to understand the nonlinearity between the contributed variables as well as the compressive strength of concrete. e statistical metrics and the graphical presentations established the robustness          60 70 80 (f )     Complexity of the developed DLNN model for predicting the compressive strength of EF concrete. e proposed model indicated a robust machine learning model that can be implemented in actual practice to calculate the compressive strength of concrete as a prior stage for material design. is is giving credit to the application of computeraided models to take place of an essential alternative technology. For future studies, the advanced models of AI can be introduced to improve the predictability of standalone AI models for compressive strength prediction of EF concrete such as [86,87]. e application of uncertainty analysis can be examined to investigate the efciency of input variables and the sensitivity of the developed model [88].

Conclusion
Accurate prediction of the compressive strength of EF concrete is very important in the sustainable development of concrete structures. Four AI approaches called DLNN, MARS, ELM, and RF were developed using 353 experiment samples collected from open-source literature. ree data division scenarios (70%-30%, 80%-20%, and 90%-10%) were applied to investigate the e ect of changing training data size on the modelling process of compressive strength. XGBoost algorithm abstracted the most correlated input parameters needed to achieve the best prediction performance. e capacity of the developed model was examined by using statistical metrics and graphical presentations. e results re ected that the DLNN model attained the highest accuracy of prediction performance and, therefore, is suitable for the CS prediction of EF concrete. e results also demonstrated that using of 90%-10% data division scenario revealed a better prediction ability. It is found that the data size used for training the developed model has a considerable impact on the predicted model accuracy. e application of XGBoost as a feature selection approach plays an essential role in abstracting the most correlated variables in CS prediction. e study revealed that the highest prediction performance could be attained by incorporating all input variables in the prediction problem.
e results re ected that the DLNN model attains the best prediction accuracy with the sixth input combination. In addition, the study demonstrated that the DLNN model provided an outstanding ability to capture the nonlinear relationship among the input variables as well as the compressive strength of concrete.

Data Availability
Data will be available upon request from the authors.