Investigation of ANN Model Containing One Hidden Layer for Predicting Compressive Strength of Concrete with Blast-Furnace Slag and Fly Ash

e prediction accuracy of concrete compressive strength is important and considered a challenging task, aiming at reducing costly and time-consuming experiments. Moreover, compressive strength prediction of concrete using blast-furnace slag (BFS) and fly ash (FA) is more difficult due to the complex mix design of a composition. In this investigation, an approach using the artificial neuron network (ANN), one of the most powerful machine learning algorithms, is applied to predict the compressive strength of concrete containing BFS and FA. e ANN models with one hidden layer containing 13 neuron number cases are proposed to determine the best ANN structure. Under the effect of random sampling strategies and the network structures selected, Monte Carlo simulations (MCS) are introduced to statistically investigate the convergence of results. Next, the evaluation of the model is concluded over 100 simulations for the convergence analysis. e results show that ANN is a highly efficient predictor of the compressive strength using BFS and FA, with maximum values of the coefficient of determination (R), root mean square error (RMSE), and mean absolute error (MAE) of 0.9437, 3.9474, and 2.9074, respectively, on the training part and 0.9285, 4.4266, and 3.2971, respectively, for the testing part. e best-defined structure of ANN is [8-24-1], with 24 neurons in the hidden layer. Partial Dependence Plots (PDP) are also performed to investigate the dependence of the prediction results of input variables used in the ANN model. e age of sample and cement content are found to be the two most crucial factors that affect the compressive strength of concrete using BFS and FA. e ANN algorithm is practical for engineers to reduce costly experiments.


Introduction
In view of the global sustainable development, supplementary cementitious materials (SCM) need to be used for cement replacement in the concrete industry. e most worldwide available SCM are fly ash (FA), a fine powder and a by-product of burning pulverized coal in electric generation power plants, and blast-furnace slag (BS), a by-product of iron ore processing. In the current context, the developed industry generates a large amount of industrial waste and seriously affects the environment. Amongst various byproducts generated by the industries, FA and BFS are of great interest to concrete researchers. Taking advantage of these materials will contribute to reducing environmental pollution and be also a cost-effective solution for producing concrete. Besides, the use of BFS and fly ash in concrete as a partial cement replacement could significantly improve concrete properties, such as compressive strength and permeability of concrete, the durability of concrete [1][2][3][4], and the workability of concrete [5]. For these reasons, the determination of BFS and FA contents for concrete mix design is essential and meaningful, especially in improving the compressive strength of concrete.
Numerous experimental studies have been conducted to determine the BFS content in the concrete mix design. Oner and Akyuz [6] have proved that the content of BFS to maximize the strength is about 55-59% of the total binder content. Shariq et al. [7] have studied the effect of BFS content on the concrete compressive strength using 20%, 40%, and 60% of BFS and three different water-to-cement (W/C) ratios. e compressive strength of concrete containing 40% BFS is higher than those containing 20% or 60% of BFS for all W/C ratios. Siddique and Kaur [8] have concluded that 20% of cement replaced with BFS can be used appropriately in structures resistant to high temperature. Tüfekçi and Çakır [9] have shown that the compressive strength of concrete using 60% BFS content reached the highest value at 28 days. Besides, Majhi et al. [10] have experienced the highest concrete compressive strength by using 40% BFS replacement.
Moreover, many experimental investigations for determining the BFS and FA replacement content in concrete mix design have been performed. Gehlot [11] has evaluated the compressive strength of concrete containing BFS and FA with different BFS/FA weight ratios, such as 0/0, 10/20, 20/ 10, 30/0, and 0/30. It has been found that the higher the weight ratio of BFS/FA, the higher the compressive strength of concrete. Li and Zhang [12] also experimented with the compressive strength of concrete using FA and BFS. e accuracy of the compressive strength prediction is strongly dependent on the number of experimental tests and the range of the mixture composition content. erefore, a new approach needs to be developed for reducing the timeconsumed and experimental cost due to a high number of experimental tests. Also, a universal prediction approach with high prediction accuracy needs to be used.
In recent years, Artificial Intelligence (AI) has been widely used for modeling many problems in the areas of science and engineering [13][14][15][16][17]. AI approaches have been developed to predict different properties of concrete, such as the shear strength of reinforced concrete beams [18,19], corrosion of concrete sewers [20], crack width of concrete [21], the ultimate strength of reinforced concrete beams [22], strength of recycled aggregate concrete [23], the compressive strength of silica fume concrete [24], compressive strength of geopolymer concrete [25], compressive strength prediction of concrete using BFS [26][27][28][29][30][31], or concrete using FA [32][33][34][35]. Among AI algorithms, ANN is currently the most powerful algorithm to simulate complex technical problems [36,37]. ANN model is capable of solving complex, nonlinear problems and especially problems in which the relationship between the inputs and outputs is not easily established explicitly. As an example, Bilim et al. [30] have used 225 data samples with six input parameters (including cement content, ground granulated blast-furnace slag content, water content, superplasticizer, aggregate content, and the age of samples) for the development of the ANN model to predict the compressive strength of concrete. e best value of the coefficient of determination for this ANN model is equal to R 2 � 0.96. Besides, in using 204 data samples, Chopra et al. [38] have proposed an ANN model containing one hidden layer with 50 neurons to predict the compressive strength of concrete using BFS and FA. e performance of such an ANN model is evaluated by a coefficient of determination of 0.92. Besides, Yeh [39] has used the highest number of data with 990 data samples to develop an ANN model for predicting the compressive strength of concrete containing BFS and FA. In the investigation of Yeh [39], the ANN structure containing one hidden layer with eight neurons is proposed, the accuracy of the ANN model is relatively high with the highest coefficient of determination of R 2 � 0.922. Overall, the performance of the ANN model depends significantly on the database (such as the number of data samples or the range distribution of variables) and the ANN structure, reflected by the number of hidden layers and number of neurons in each hidden layer [40]. erefore, the determination of neuron number and the hidden number is crucial for increasing ANN performance. erefore, the primary purpose of this investigation is to propose an efficient ANN model to improve the compressive strength prediction performance of concrete containing BFS and FA, thanks to a significant number of data gathered from the literature. Furthermore, the efficiency of the proposed ANN model is determined by (i) determination of neuron number for one hidden layer using empirical formulations proposed in the literature, (ii) investigation on the results convergence of each ANN structure, (iii) evaluation of the prediction performance of each model to determine the best ANN structure, and (iv) using the best ANN structure for predicting the compressive strength of concrete. e best ANN architecture is evaluated through three statistical measurements, namely, the coefficient of determination (R 2 ), mean absolute error (MAE), and root mean square error (RMSE). Finally, a sensitivity analysis using Partial Dependence Plots (PDP) is performed to evaluate the influence of each input variable on the prediction of compressive strength of concrete containing BFS and FA.

Research Significance
Predicting concrete compressive strength using supplementary cementitious materials, such as BFS and fly ash, with high accuracy and reliability plays a crucial role in many civil engineering applications. Although this research topic has been the subject of intense researches over the past two decades (i.e., Yeh [39], Han et al. [41], Boukhatem et al. [27], Kandiri et al. [28], Boga et al. [29], Behnood et al. [42], Dao et al. [43], and Bui et al. [44]), there are still problems that need to be dealt with. First of all, the limited number of data points and range of input parameters used to construct ML models put strong constraints that inhibit the applicability of these models from an engineering point of view. Second, the reliability of ML models in predicting compressive strength requires a rigorous assessment approach.
ird, from a practical point of view, the efficiency of an algorithm, otherwise, the total computation time, should be considered the most important factor. erefore, the present work attempts to address the above-mentioned gaps with the following ideas: (1) To the best of the authors' knowledge, the secondlargest dataset containing 1274 data points is used, in which the collecting process is carefully conducted, and duplicate samples are removed from the database (2) e prediction performance of different one hiddenlayer ANN architectures are evaluated, using semiempirical formulas suggested in the relevant 2 Advances in Materials Science and Engineering literature to determine the appropriate number of neurons (3) Only single hidden layer ANN models are considered and developed, with the highest aim to promote simplicity and boost efficiency (4) e reliability of ANN models is rigorously assessed by Monte Carlo simulations (5) e predictability of the best architecture is shown more relevant compared with 11 investigations published in the literature and clearly confirms the simplicity and effectiveness of the proposed ANN model

Database Construction
In this study, 1274 data samples of experiments on compressive strength of concrete containing BFS and FA are rigorously gathered from 4 other investigations (cf. Table 1), including 10 samples from Pitroda [33], 204 samples from Chopra et al. [38], 990 samples from Yeh [39], and 72 samples from Lee et al. [45]. Different from most of the works published in the literature, 40 duplicate data points from the work of Yeh [39] are filtered out of the original 1030 instances, as this might affect the accuracy and reliability of prediction results. e database includes eight input variables, namely, the cement content (I 1 ), water content (I 2 ), coarse aggregate or gravel content (I 3 ), fine aggregate or sand content (I 4 ), blast-furnace slag content (I 5 ), fly ash content (I 6 ), superplasticizer content (I 7 ), and age of samples (I 8 ), along with one output variable, the compressive strength (CS) of concrete. Table 1 summarizes the database, including the number of data samples collected in each reference and their percentages of proportion. Figure 1 describes the database distributions. It can be seen that most of the input variables in the database possessed a wide range of values. e cement content is 100 ÷ 610 (kg/m 3 ). e water content is mainly in the range of 150 ÷ 200 (kg/m 3 ). e coarse aggregate or gravel content is in the range of 800 ÷ 1200 (kg/m 3 ), with a few values higher than 1200 kg/m 3  e corresponding correlation analysis with the f c is displayed in Figure 2. As clearly shown, none of the variables are significantly correlated. e highest linear correlation coefficient is equal to 0.54 between input I 1 and f c . erefore, the eight inputs are relatively independent of each other and could be used as input variables for the prediction problem.

Simulation Using Neural Networks
ANN is a powerful machine learning-based data analysis algorithm, which is a model of bioneural networks. is machine learning approach attempts to simulate the process of knowledge acquisition and inference occurring in the human brain [46]. ANN has been widely used to address nonlinear regression analysis problems. Backpropagation neural network (BPNN), a standard training method of ANN, is often used for regression analysis and practical applications [47]. A backpropagation network structure is a combination of different layers, where the first layer is the input layer, the last layer is the output layer, and the middle layers are hidden layers connected to both the input and output layers. Typical backpropagation networks typically use a gradient descent algorithm like Widrow-Hoff arithmetic. In this network, weights are changed or moved along the negative value of the gradient of the executing function. e term backpropagation is used because it relates to how the gradual computation of nonlinear multilayer neural networks is performed. In practice, to design or use backpropagation neural networks to learn or train linear networks to solve a particular problem, the following basic steps are usually performed: (a) practicing aggregate learned or trained data; (b) building neural networks; (c) network training; and (d) application of neural networks to simulate new data. e block diagram of the backpropagation network is shown in Figure 3.

Number of Hidden
Neurons. An important issue in designing a network is how many neurons are needed in each hidden layer. Using too few neurons can lead to either incomplete signal recognition in a complex dataset or underfitting. Using too many neurons increases the lattice time, perhaps too much to train when it is impossible to train in a reasonable amount of time. A large number of neurons can lead to overfitting, in which case the network has too much information, or the amount of information in the training set does not have enough specific data to train the network [40]. e best number of hidden units depends on many factors-the number of inputs, outputs of the network, the number of cases in the sample set, the noise of the target data, the complexity of the error function, network architecture, and network training algorithm.
In the majority of cases, there is usually no way to easily determine the optimal number of neurons in the hidden layer without having to train the network [48]. e best way  is to use the trial-and-error method. In fact, it is possible to use the forward selection or backward selection method to determine the number of units in the hidden layer. Progressive selection begins with choosing a reasonable rule for the performance evaluation of the network. After that, a small number of hidden units, train, and test are chosen and then evaluate the network performance. After that, slightly increase the number of hidden units and conduct and retry until the error is acceptable, or there is no further significant improvement. e backward selection, in contrast to the forward selection, starts with a large number of units in the hidden layer and then descends. is process is very timeconsuming but helpful in finding the right number of units for the hidden layer.

Neural Network Evaluation Procedure.
e disadvantage of the backpropagation algorithm is that its convergence speed is relatively slow [49]. erefore, many powerful optimization algorithms have been used, most of which have been based on simple gradient descent algorithms. One of the algorithms to improve the convergence rate or the learning rate of the neural network is the backpropagation training network according to the Conjugate Gradient algorithm.
In conjugate gradient algorithms, the search direction for all conjugate gradient algorithms is occasionally reset to the gradient's negative [50]. When the number of repetitions is equal to the number of network parameters, namely, weights and bias, the standard reset point occurs. To improve the efficiency of training, there have been other reset 0.14 0.  Advances in Materials Science and Engineering approaches. In those approaches, Powell [51], based on an earlier version proposed by Beale [52] which suggests the technique will restart if there is very little orthogonality left between the current gradient and the previous gradient. If very little orthogonality remains between the current and the previous gradient, the technique will restart. is is tested with the following inequality: where g k is the gradient of the k th iteration. If this condition is satisfied, the search direction is reset to the negative of the gradient.

Validation of Models.
Evaluating the model's accuracy is an essential part of the machine learning modeling process to describe the model's performance level in its predictions. In this study, three measures of statistical performance, namely, root mean square error (RMSE), mean absolute error (MAE), and coefficient of determination (R 2 ), are used to evaluate the difference between results of each network and its ability to make accurate predictions. In general, these criteria are popular for quantifying the performance of machine learning algorithms. More specifically, the mean squared difference between the real and estimated determines the RMSE, while the average magnitude of the errors determines the MAE. R 2 evaluates the correlation between the actual and estimated values. Quantitatively, the lower RMSE and MAE values indicate better performance of the models. In contrast, higher R 2 values indicate better model performance. RMSE, MAE, and R 2 are estimated as follows: where a j is the actual value; a ⌢ j is the predicted value; a j is the average of actual values; and N is the total number of samples.

Methodology Flowchart
e methodology of this study includes three main steps as follows: (i) Preparation of the data: in this step, the collected dataset is randomly divided into two parts: the first part accounts for 70% of the data and is used to train the network. e second part is the remaining 30% of the data and used to test the network's performance.
(ii) Model building and training: in this step, the training dataset is used to construct the ANN model. From there, the appropriate structure of the ANN model is selected. (iii) Model validation: in this step, the trained models are tested and validated using the testing dataset. e predictability of the proposed model is assessed through statistical criteria such as R 2 , RMSE, and MAE.
A schematic diagram of the methodology is illustrated in Figure 4.

Number of Neurons in a Single Hidden Layer of ANN.
In this section, the number of neurons in a hidden layer is determined through several formulas proposed in the literature. Eighteen formulas are collected and summarized in Table 2. In the 18 formulas, the twelve formulas are based on the input variables to determine the number of neurons in one hidden layer. e six remaining formulas are based on both input variables and output variables to determine the number of neurons in one hidden layer. Based on eight input variables and one output variable, 18 values of neurons are determined and shown in Table 2. It is worth noticing that the adjacent integer values are taken if the calculated values are not an integer. By doing so, 25 values are determined using 18 formulas. ese values are distributed in the range from 1 to 255 and displayed in Figure 5. Figure 5 depicts the neuron number for 24 cases, except for the case with 255 neurons (not shown in Figure 5 for better illustration). By removing similar values, a total of 13 values are identified for use in the ANN model. Besides, the basic parameters of ANN used in this study are presented in Table 3, including fixed parameters and neuron numbers in the hidden layer. e number of inputs is equal to 8, and the only compressive strength of concrete is considered the output.
e sigmoid function is used as the activation function of the hidden layer, and the linear function is used as the activation function of the output layer. e training algorithm with conjugate gradient backpropagation with Powell-Beale restarts is used.

Prediction Performance and Statistical Analysis.
In this section, performance criteria of the ANN model with 13 different cases of neuron number are shown in Figure 6, including (a) coefficient of determination (R 2 ), (b) root mean square error (RMSE), and (c) mean absolute error (MAE). A total of 1300 simulations are conducted. e statistical measurements of the simulations are shown in Figure 6, highlighting the mean values and standard deviation (Std) over 100 simulations.
It can be seen that the maximum value of R 2 obtained for the training dataset is R 2 � 0.967, and the minimum value is R 2 � 0.667. Besides, the maximum value of R 2 obtained for the testing datasets is R 2 � 0.883, and the minimum value is R 2 � 0.665. e RMSE values range from 3 to 9.5 for the training parts, and from 5.75 to 11.7 for the testing parts. e results also indicate the MAE values are in the range of 2 ÷ 7 (training datasets) and 3.9 to 7.1 for the testing datasets.
e Std values of each case also are displayed in Figure 7.    Table 4 shows four values of quality assessment criteria (maximum, minimum, average, and Std) over 100 simulations with three ANN-16N, ANN-24N, and ANN-33N      ANN-24N is the most reliable model with the highest performance. However, before any further conclusion, the convergence analysis of these three ANN models needs to be evaluated.

Investigation on the Convergence of Prediction Results.
e use of Monte Carlo simulation for convergence analysis on the results is important, aiming at determining (i) the suitable number of Monte Carlo simulations and (ii) the reliability of the prediction results. Figure 7 depicts the convergence of results for three ANN architectures proposed in this investigation, namely, the ANN models with 16, 24, and 33 neurons. It is worth noting that the convergence is performed for both training and testing datasets for all cases over 100 simulations. Figures 7(a)-7(c) show the convergence curves of R 2 , RMSE, and MAE, respectively. It can be observed that these values are relatively stable after about 50 simulations for both the training and testing parts. It could be stated that the results obtained by the proposed ANN model with a different number of neurons in the hidden layer are converged, even under the random sampling effect.
Regardless of the training phase of the ANN model, the performance analysis is focused on the testing parts, as the latter directly reflect the prediction performance of machine learning models. It can be seen that the 24-neuron ANN  erefore, it could be concluded that the ANN model with 24 neurons is the best architecture for predicting the f c of concrete.

Prediction Performance of Typical ANN Architecture.
is section is dedicated to the presentation of typical prediction results of the best ANN architecture containing 24 neurons in a single hidden layer. e correlations between the predicted and the experimental values are shown in Figures 8(a) and 8(b) for the training and testing part, respectively, through a regression model. e plot of a linear fit is performed in each case, represented by a continuous blue line. Figure 8 demonstrates a high correlation between the experimental and predicted compressive strength of concrete using BFS and FA. Figures 9(a) and 9(b) show the probability distribution of errors for the training and testing datasets of the best ANN model, respectively. Figure 9(a) depicts that the ANN-24N can successfully predict the compressive strength of concrete for the training set, where the prediction error is relatively low. Almost error prediction is equal to about 0 with 370 samples for the training part, and 160 samples for the testing part. For the testing part, in only two prediction cases, the errors are found high, with an absolute value equal to 20 MPa. Table 5 shows different values of performance criteria for the best architecture of the ANN model including 24 neurons. e best values of R 2 are 0.9437 and 0.9285 for the training part and for testing part, respectively. e values of RMSE, MAE, Err. Mean, and Err. Std for the training dataset are 5.4480, 4.1365, −0.0563, and 5.4563 and, for the testing dataset, are 4.9585, 3.9423, 0.6252, and 4.9647, respectively. Table 6 shows the comparison of different machine learning models proposed in the literature with the ANN model of this investigation. e comparisons are presented in the form of the machine learning algorithm, input number, number of data, and performance measure. e results show the ANN model of this investigation, using only a single hidden layer, could predict the compressive strength of concrete with high reliability and higher accuracy than almost all investigations.
First, it is important to notice that high prediction accuracy is reported with a low number of samples in the original database. For instance, Sarıdemir et al. [31], Boga et al. [29], Han et al. [41], Bilim et al. [30], and Kandiri et al. [28] have reported the values of R 2 � 0.981, 0.971, 0.961, 0.960, and 0.9409, respectively, with the data points in the range of 162 to 624. e testing age of concrete samples, which has been found the most influencing parameter of concrete compressive strength [43], has not been considered as input variables in [29,41] and [31]. Besides, fly ash has not been considered in the works of Kandiri et al. [28] and Bilim et al. [30].
Second, regarding the works of Boukhatem et al. [27], Bui et al. [44], Golafshani and Behnood [69], and Dao et al. [43], more data points are considered. However, the prediction accuracy of this work is greater (R 2 � 0.9285) compared with the reported values of R 2 , ranging from 0.8806 to 0.9216. Notably, the database in this study uses more samples than the studies mentioned earlier.
ird, the results in Feng et al. [70] reach a higher R 2 value (R 2 � 0.9820). However, it is noted that this contribution used 8 inputs with 1030 samples in Yeh's work, including 40 duplicate data points. Moreover, 90% of data are used to train the model, the remaining data to test the prediction accuracy so that the reported accuracy only corresponds to the 10% remaining data. In this study, the classical 70-30 train-to-test ratio is used, which puts more constraint on the model's predictability by covering a broader range of input values and more concrete samples on the testing phase.
Similarly, Behnood et al. [42] have used the highest sample amount of concrete in the literature (1912 samples).
is study shows the proposed ANN model can predict the concrete compressive strength with R 2 � 0.9285, which is greater than that of Behnood et al. [42] (R 2 � 0.90). It is worth noticing that the authors have used a train-to-test ratio of 85/15, compared with a 70/30 ratio of this study.
Overall, these comparisons have confirmed the high accuracy of the proposed ANN model in ensuring prediction reliability. Moreover, the single hidden layer ANN model clearly shows its simplicity and efficiency in total computation time than other hybrid machine learning approaches.
ese results indicate that if the architecture of an ANN model is carefully selected, it could be effectively used as an alternative prediction tool for material engineers.
6.5. Sensitivity Analysis. In this section, PDP analysis is performed and estimated for eight variables, which correspond to 8 input variables used in the ANN model, namely, On the basis of the PDP values, the effect of input variables on the compressive strength of concrete is most pronounced with the age of samples, followed by cement, fly ash, water, coarse aggregate, superplasticizer, blast-furnace slag, and fine aggregate contents. is order depicts that the most critical input effect is the age of samples. Further, PDP investigation shows that most of the effects are positive,      except for superplasticizer content with a negative effect. It is interesting to notice that the negative effect of superplasticizer is also proved by an experimental investigation of Benachai et al. [71]. Besides, the PDP investigation shows that the optimum water content is equal to 150 kg/m 3 . With higher cement and BFS contents, the compressive strength increases. With an important number of data samples distributed from 75 to 200 kg/m 3 , the result of the PDP investigation could be considered reliable in this range, where the compressive strength is in the range of 35 to 65 MPa. Finally, it could be observed that the compressive strength strongly increases from 1 to 28 days. After this period, the compressive strength continues to develop, but at a slower rate than the previous strength development.

Conclusion
In this investigation, a well-known machine learning ANN algorithm has been introduced to predict the compressive strength of concrete containing blast-furnace slag and fly  ash. A number of 1274 experimental results have been gathered to construct a database and develop the ANN model. In this database, 70% of data is randomly chosen for the training phase of ANN, and 30% of the remaining data is used for the testing phase of the ANN model. Monte Carlo simulation is performed to determine the necessary number of simulations for obtaining converged prediction results, in which 100 simulations are proven a sufficient number of runs. e analysis shows that the ANN-24N (24 neurons in a single hidden layer) is the most stable model and produces the best prediction performance. e values of R 2 , RMSE, and MAE of the best model are, respectively, 0.9285, 4.4266, and 3.2971 for the testing part. Partial Dependence Plots (PDP) analysis is used to investigate the dependence of prediction results of eight input variables in this study. e age of samples and the cement content are determined to be the two most important parameters affecting the compressive strength of concrete. e results of this investigation could help in constructing a reliable soft computing tool to predict promptly and quickly the compressive strength of concrete containing blast-furnace slag and fly ash (see supplementary materials). Once such a tool is carefully built, the prediction process can reduce the time consumption and cost of experimental tests. e limitation of the present work might be the ranges of inputs and output of the database. erefore, these ranges would limit the applicability of the ANN model, and also the numerical tool in the supplementary materials. To improve the prediction accuracy and reliability of the ANN model, a new database should be developed, which is the short-term research direction of the present work.
Data Availability e processed data are available from the corresponding author upon request.