Mathematical Problems in Engineering
Volume 2021 (2021), Article ID 5570945, 17 pages
https://doi.org/10.1155/2021/5570945

Research Article

Evolution of Deep Neural Network Architecture Using Particle Swarm Optimization to Improve the Performance in Determining the Friction Angle of Soil

Tuan Anh Pham, Van Quan Tran, and Huong-Lan Thi Vu

University of Transport Technology, Hanoi 100000, Vietnam

Correspondence should be addressed to Tuan Anh Pham; anhpt@utt.edu.vn

Received 19 February 2021; Accepted 23 April 2021; Published 7 May 2021

Academic Editor: Venkatesan Rajinikanth

Copyright © 2021 Tuan Anh Pham et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract.

This study focuses on the use of deep neural network (DNN) to predict the soil friction angle, one of the crucial parameters in geotechnical design. Besides, particle swarm optimization (PSO) algorithm was used to improve the performance of DNN by selecting the best structural DNN parameters, namely, the optimal numbers of hidden layers and neurons in each hidden layer. For this aim, a database containing 245 laboratory tests collected from a project in Ho Chi Minh city, Vietnam, was used for the development of the proposed hybrid PSO-DNN model, including seven input factors (soil state, standard penetration test value, unit weight of soil, void ratio, thickness of soil layer, top elevation of soil layer, and bottom elevation of soil layer) and the friction angle was considered as the target. The data set was divided into three parts, namely, the training, validation, and testing sets for the construction, validation, and testing phases of the model. Various quality assessment criteria, namely, the coefficient of determination (R²), mean absolute error (MAE), and root mean square error (RMSE), were used to estimate the performance of PSO-DNN models. The PSO algorithm showed a remarkable ability to find out an optimal DNN architecture for the prediction process. The results showed that the PSO-DNN model using 10 hidden layers outperformed the DNN model, in which the average correlation improvement increased R² by 1.83%, MAE by 5.94%, and RMSE by 8.58%. Besides, a global sensitivity analysis technique was used to detect the most important inputs, and it showed that, among the seven input variables, the elevation of top and bottom of soil played an important role in predicting the friction angle of soil.

1. Introduction

In geotechnical engineering, the internal friction angle of soil is one of the most important parameters [1]. This index is mentioned in many studies in the field of geology, for instance, in the investigation of the foundation bearing capacity [2–7], soil stability [3, 8–10], limit loads [11], and slope stability [9, 12–14]. In slope stability or retaining walls-related problems, the friction angle was proven to affect the soil slope stability significantly, as an increase of the friction angle decreases the soil consolidation [3]. Similarly, the retaining wall’s reaction is also significantly affected by the internal friction angle [8]. In another study, You Mo et al. [11] pointed out a positive correlation between friction angle and critical load. The above points showed that the friction angle is a vital soil parameter [1], and more in-depth research of the latter is required. However, the friction angle is only mentioned indirectly in related geological studies [2–4, 8, 9, 11], and there are limited researches on this index.

The value of the internal friction angle depends on many soil parameters, such as grain size, particle shape, unit weight, specific gravity, and soil moisture [15, 16]. Two main approaches were proposed for estimating the friction angle of soil: experimental and theoretical approaches. Concerning the experimental one, the internal friction angle can be obtained from the direct shear test [1, 17–19] and the triaxial shear test [4, 19–22]. However, these experiment testing methods possessed some disadvantages, as they are time-consuming and required relatively expensive equipment [23–25]. Besides, theoretical and numerical methods were also applied to determine the friction angle of soil. As an example, Mo et al. [11] proposed an equation for the friction angle of the soil and confirmed the validity of the theoretical method.

Notwithstanding the foregoing, the theoretical method’s limitation is that it is necessary to be based on some assumptions, in this case, the ideal rigid-plastic slip-line field theory. Other studies proposed alternative methods to evaluate the internal friction angle of soil, based on the standard penetration test (SPT) values [26–29] and cone penetration test (CPT) results [30, 31]. Those methods provided several empirical formulas that could quickly predict the soil friction angle. However, these formulas contained only one or several main input variables, which could reduce the prediction accuracy in many cases.

In the last two decades, based on the development of computer science, numerous studies have proposed a novel approach for surveying and assessing geological issues, namely, the artificial intelligence (AI) approach [10, 25, 32, 33]. In Mikaeil et al.’s work [32], the authors have used the genetic algorithm (GA) to study three crucial physical and mechanical characteristics of soil, in which the internal friction angle is an important parameter. Additionally, an adaptive neuro-fuzzy inference system (ANFIS) has been used by Murlidhar et al. [25] to study the shear strength of rock based on the internal friction angle. Das and Basudhar [34] and Al-Hamed et al. [35] have developed ANN models to predict the internal friction angle of soil. Besides, Pham et al. [36] have developed a hybrid model using random forest and particle swarm optimization for estimation of undrained shear strength of soil. The obtained results showed the effectiveness and reliability of AI approach, reflected by the high R² values achieved [10, 25, 32, 33]. Besides, Deep Neural Network (DNN) is also another powerful and efficient algorithm of AI. Nowadays, DNN has become popular and widely used to solve practical engineering problems and provide reliable results [37–40]. The problem with such an ML approach is that it has a large number of critical hyperparameters, making it difficult to find an optimal model architecture. Among various optimization algorithms, Particle Swarm Optimization (PSO) is an efficient, robust, and straightforward algorithm mainly used to solve problems that are difficult to find an exact mathematical model [41–43].

Therefore, the main objective of this study is to develop a PSO-DNN hybrid model, which can be self-developed to find the best architectural model to predict the internal friction angle of the soil, including the number of hidden layers and number of neurons in each hidden layer. Seven input factors that might affect the prediction of friction angle of soil were considered: the soil state, standard penetration test value, unit weight of soil, void ratio, thickness of soil layer, top elevation of the soil layer, and bottom elevation of the soil layer. For this aim, a data set consisting of 245 soil samples was collected from some drill holes in Ho Chi Minh City, Vietnam. The database was then divided into the training (60% of the data set), validation (20% of the data set), and testing (20% of the data set), related to the training, validation, and testing phases of the hybrid model. In addition, a global sensitivity analysis method using Monte Carlo simulation was conducted to find out the most important parameter that affects the prediction of friction angle of soil.

2. Data Collection and Preparation

2.1. Experimental Measurement of Friction Angle of Soil

In this work, the authors conducted 245 identified different soil samples in Ho Chi Minh City, Vietnam (Figure 1), and the results are summarized in Table 1. To determine the characteristics of the soil properties, we use the method of determination of shear laboratory resistance in a shear box apparatus [44, 45]. The test samples with the original structure and natural moisture were prepared by cutting from the original soil samples into blocks, which were taken into the ring knives by the method of determining the volume by the ring knife [46]. The underside and top of the soil sample should be leveled with the circular knife's edge and placed with dampened paper first. For fast cutting without draining, the absorbent paper must be replaced with tracing paper (or thin plastic) [46]. Simultaneously with cutting soil samples, it is necessary to take soil to determine moisture content.

Figure 1: Experimental location (). Source (): CIA Map.

Table 1: Inputs and output of the present study.

N°
S
N30
G
e
H
Z₁
Z₂
(ϕ)

Unit
(kN/m³)
(m)
(m)
(m)
(o)

1
3
8
18.95
0.846
3.600
4.300
7.900
16.983

2
4
8
19.33
0.782
3.400
1.100
4.500
20.042

3
4
17
19.47
0.765
14.000
1.200
15.200
22.328

4
3
8
18.9
0.842
3.100
3.200
6.300
15.600

.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.

243
3
5
18.68
0.930
6.600
2.000
8.600
8.870

244
6
16
20.63
0.560
5.400
4.000
9.400
16.420

245
6
44
20.41
0.584
13.100
40.000
53.100
16.902

Min
2
2
16.2
0.416
0.800
0.000
2.000
4.23

Average
4.649
14.629
19.24
0.784
6.305
13.369
19.673
16.648

Max
9
85
21.57
1.394
33.000
77.320
79.320
33.4

SD
1.882
10.940
0.948
0.194
5.560
14.414
15.590
6.953

With sandy soil, the sample was prepared by pouring dry sand onto a cutter installed in a hard bottom box with many small holes. Then, take a sample into a cylindrical ring knife with two ends, place the sample in the cutting box, and compress the sample under pressure σ. If soil does not preconsolidate, cut samples immediately, cut quickly, and cut samples at a speed of 1 mm/minute until the sample is damaged. If the soil consolidates under pressure σ, maintain that compressive force until the level of consolidation is met, and then proceed as normal. It is required to cut slowly at a cutting rate of 0.01 mm/min (or slower) until the sample breaks. The destructive force is the maximum value read on the strain gauges. The vertical compression pressure was applied at four levels: 50 kPa, 100 kPa, 200 kPa, and 300 kPa, respectively. For each level of compressive pressure, the corresponding shear strength of soil was recorded. Draw the shear strength according to the vertical pressure, thereby determining the cohesion of soil C, tgϕ and calculating the internal friction angle (ϕ).

2.2. Data Preparation

To predict the internal friction angle of the soil, the inputs need to be carefully considered. First, the water content was not included in this study, as it is believed that the effect was taken into account in the number of SPT blows [47]. Besides, many studies have been successful in finding the relationship between the results of in situ experiments (CPT and SPT) and the Plastic and Liquid Limited humidity [48–51]. Since the SPT is one of the most commonly used tests in practice to indicate the soil's ability to withstand compression and in situ shear, it is believed to relate to the factors that characterize the ability of the shear strength of the soil. Many studies have shown the relationship between the in situ experiment results and the internal friction angle of the soil [26–31]. Second, this study aims to provide a tool to quickly analyze the internal friction angle of the soil based on basic parameters that can be quickly and simply determined instead of parameters that take time and cost for determination. Therefore, this study selected SPT value (N30) of soil as the main feature, and some other indicators such as soil state (S), unit weight of soil (G), void ratio (e), thickness of soil layer (H), top elevation of soil layer (Z₁), and bottom elevation of soil layer (Z₂) are inputs to predict the internal friction angle of soil. The internal friction angle of soil (ϕ) is the single output variable.

One of the most popular methods to determine the internal friction angle of soil (ϕ) is the direct shear test method [44, 45]. There are three soil direct cutting modes: UU mode (this mode is fast cutting-no cohesion, no drainage), CU mode (fast cutting, cohesive-no drainage), and CD mode (slow cutting, consolidation-drainage; this mode usually applies to sandy soils) [1]. Firstly, the UU cutting mode is suitable for fast construction and difficult drainage, and the soil samples in the present study were all under this type of condition. Secondly, the UU cutting mode is quick test, taking under 30 minutes to perform, while the CU and CD modes take longer to execute and can take weeks or even months to complete. Besides, the shear strength parameters given by the UU cutting mode are generally safer than those in the other two modes. Therefore, in this study, the results of the UU cutting mode were applied to determine the friction angle of all soil samples. The meaning of the soil type state number is explained in Table 2. Besides, the parameters’ meanings are illustrated in Figure 2, and the histograms of all the input and output variables are shown in Figure 3.

Table 2: Soil state number explanation.

Soil type
Soil state number
Soil type-state

Clay
1
Very soft clay

2
Soft clay

3
Firm clay

4
Stiff clay

5
Very stiff clay

6
Hard clay

Sand
7
Loose sandy

8
Medium sandy

9
Dense sandy

Figure 2: Diagram for parameters.

(a)

(b)

(c)

(d)

(e)

(f)

(g)

Figure 3: Histograms of the data set in this study. (a) S. (b) N30. (c) G. (d) H. (e) Z₁. (f) Z₂. (g) ϕ (°).

The data set containing 245 samples was statistically introduced and summarized in Table 1, including several soil samples, min, max, average, and standard deviation of the input and output variables. It can be seen that, among 245 soil samples, 46 samples are sandy soil and the rest are clayey soil. Soil state number (S) ranged from 2 to 9, corresponding to soft clay to dense sandy soil. The minimum depth of the soil samples is 2.0 m, and the maximum depth is 79.32 m. The standard penetration test values (N30) were from 2 to 85, where 2 corresponds to the soft clay soil and 85 corresponds to the hard clay soil. The unit weight of soil (G) ranged from 16.2 (kN/m³) to 21.57 (kN/m³) in accordance with the clay soil to sandy soil layers. The void ratio (e) varied from 0.416 to 2.194, corresponding from medium sandy soil to soft clay soil. The internal friction angles of soil ranged from 4.23 (degrees) to 33.4 (degrees), related to the shear strength of the soil increasing from soft clay to medium and dense sandy soil. These input data results are consistent with the properties of the soil layers. An example of 50 data samples is given in Table 3.

Table 3: An example of data set of the present study.

N°
S
N30
G
e
H
Z₁
Z₂
ϕ

Unit
kN/m³
M
m
m
(degree)

1
3
8
18.95
0.846
3.6
4.3
7.9
16.983

2
4
8
19.33
0.782
3.4
1.1
4.5
20.042

3
4
17
19.47
0.765
14
1.2
15.2
22.328

4
3
8
18.90
0.842
3.1
3.2
6.3
15.6

5
7
10
18.82
0.790
4.3
7
11.3
28.18

6
3
8
19.70
0.685
3
1.1
4.1
17

7
3
8
19.70
0.685
3
1.1
4.1
17

8
4
9
19.44
0.796
3.1
0.5
3.6
18.73

9
3
7
18.61
0.887
4.7
3.7
8.4
14.85

10
3
3
19.10
0.806
2.3
12.7
15
15.295

11
3
3
19.10
0.806
2.3
12.7
15
15.295

12
8
50
21.28
0.431
1
11
12
33.4

13
2
4
17.80
1.193
0.8
3.2
4
8.02

14
2
4
17.80
1.193
0.8
3.2
4
8.02

15
4
15
19.18
0.781
3.5
10.5
14
18.2

16
2
2
17.10
1.272
1.5
3.9
5.4
7.03

17
8
24
20.12
0.543
9.5
25.9
35.4
32.8

18
4
13
20.10
0.585
26.4
8.6
35
24.17

19
4
13
20.10
0.585
26.4
8.6
35
24.17

20
8
15
20.90
0.457
6.1
9.4
15.5
32.57

21
3
4
18.98
0.829
4.5
2
6.5
14.533

22
3
4
18.98
0.829
4.5
2
6.5
14.533

23
8
24
19.70
0.690
2.7
28.8
31.5
29.3

24
4
9
19.44
0.752
5.3
0.7
6
18.1

25
8
22
20.20
0.571
6.9
33.1
40
31.03

26
4
19
20.20
0.564
30.15
6.7
36.85
24.2

27
8
26
20.21
0.573
6.4
33.6
40
30.855

28
8
26
20.21
0.573
6.4
33.6
40
30.855

29
3
6
17.78
1.070
15.5
4.5
20
13.854

30
8
12
20.90
0.460
5
55
60
31.75

31
6
50
20.55
0.571
5.6
68.4
74
25.2

32
8
14
20.67
0.477
5.7
33
38.7
31.623

33
5
20
19.38
0.744
5.8
31.5
37.3
20.744

34
8
18
19.52
0.674
3.45
27
30.45
29.08

35
3
7
18.84
0.879
9.3
1.2
10.5
14.572

36
8
12
20.70
0.500
4
7.2
11.2
30.725

37
3
4
18.85
0.851
4
1.5
5.5
13.742

38
3
4
18.85
0.851
4
1.5
5.5
13.742

39
3
7
18.94
0.845
7.7
7
14.7
14.4

40
3
7
18.94
0.845
7.7
7
14.7
14.4

41
3
42
19.40
0.714
5
24.6
29.6
15

42
5
18
19.67
0.705
6.7
0.3
7
20.883

43
5
18
19.67
0.705
6.7
0.3
7
20.883

44
3
10
19.20
0.792
2
10.5
12.5
13.87

45
3
10
19.20
0.792
2
10.5
12.5
13.87

46
4
14
18.30
0.830
5.4
2.4
7.8
16.433

47
8
13
20.26
0.570
13.4
6.6
20
31.205

48
4
22
19.43
0.747
1.2
10
11.2
16.766

49
4
14
18.85
0.792
2.2
5.8
8
16.315

50
8
12
19.15
0.735
3
57
60
27.505

In this study, we divided the data into three data sets, in which the training set was used to build the model, the validation set was used to give an estimate of model skill while tuning the model’s hyperparameters, and the testing set was used to estimate of the skill of the final tuned models to choose the best model. The test set was fixed at 20% of the total data set, hidden during training, and used only to evaluate the performance of the final model after the hyperparameters tuning process.

3. Machine Learning Methods

3.1. Particle Swarm Optimization

Particle Swarm Optimization (PSO) was presented by Kennedy and Eberhart [52]. It became prevalent because it is a continuous optimization process type and allows an analysis of multiple targets. For a continuous search of the best solution, the method used is to move the positions of particles at a given velocity calculated in each iteration. Each movement of particles is influenced by its best position and the best position in the entire search space (Figure 4). This is expected to move the swarm to the best position. The effectiveness of the solution is assessed through a fitness function. In addition, PSO does not use the slope of the problem to be optimized, which means that PSO does not require optimization issues to be differentiable like standard optimization methods such as gradient descent and quasi-newton methods. PSO is a powerful technique that has been widely used for optimization issues in many fields in general and geotechnical engineering in particular [53, 54].

Figure 4: Particle movement by the swarm direction.

The pseudocode of the algorithm is presented as follows. (Algorithm 1)

FOR each particle i in swarm

    Initialize parameters: , _damp, c1, c2.

    FOR each dimension j

        Initialize position xij randomly within permissible range

        Initialize velocity randomly within permissible range

    End FOR

End FOR

Iteration k = 1

WHILE k < maximum_Iteration

    FOR each particle i in swarm

        Calculate fitness value

        IF fitness value > P_best[i] THEN

            P_best[i] = fitness value

        END IF

        IF fitness value > G_best THEN

            G_best = fitness value

    END IF

END FOR

FOR each particle i in swarm

    FOR each dimension j

            Calculate new velocity:

    (k + 1) = wvij(k) + c₁random (0,1) (P_best[i] − xij[i]) + c₂random (0,1) (G_best − xij[i])

            Update particle positon:

            xij(k + 1) = xij(k) + (k + 1)

    END FOR

END FOR

 = ._damp

k = k + 1

END WHILE

Algorithm 1Algorithm 1

The parameters in the equation that defines the velocity of the next iteration are as follows: is an inertial parameter; c₁ and c₂ are the acceleration coefficients; c₁ value gives the importance of individual best solution, and c₂ is the importance of global best candidate solution; _damp is the inertial reduction coefficient, and it helps the swarm movement to quickly converge.

3.2. Deep Neural Network

Deep neural network (DNN) is definitely one of the most advanced regression methods. DNN model is analogous to a multistage regression. The main idea is to create a flexible nonlinear statistical model consisting of several layers and neurons in each layer. In DNN, each node in a layer is associated with a certain weight, denoted as ij, with every node in the other layers creating a fully linked neural system [55]. Except for the input layer, each node is a neuron that uses a nonlinear activation function [56]. Thanks to multilayer and nonlinear activation functions, the DNN model could distinguish nonlinear separable data. The DNN structure used in this study is a multilayer perception (MLP), shown in Figure 5. Further details of the DNN model can be found in [57].

Figure 5: Illustration of the DNN network used in this study, including 10 input nodes, hidden layers, and one output node.

3.3. Particle Swarm Optimization-Deep Neural Network Algorithm

The PSO-DNN hybrid model was created based on the PSO algorithm, in which the fitness function of each particle in the swarm was DNN. The hybrid algorithm is implemented with the number of hidden layers ranging from 2 to 10. For each case of the number of hidden layers, random particles are generated, the length of the particle vector is the number of hidden layers, and the jth dimension of the particle vector is the number of hidden neurons. The swarm architecture was presented in Figure 6, in which k is the number of hidden layers and N is the number of particles in a swarm. The flow chart of the hybrid PSO-DNN model was shown in Figure 7. In this algorithm, the number of hidden layers in the model is incremented from 2 to 10. For each case of the number of hidden layers, the algorithm will automatically look for the optimal model architecture for that number of hidden layers.

Figure 6: Description of swarm architecture.

Figure 7: Flow chart of PSO-DNN algorithm.

3.4. Performance Evaluation

To evaluate the effectiveness of the proposed AI models, three widely used statistical criteria are applied, namely, mean absolute error (MAE), root mean square error (RMSE), and the coefficient of determination (R²). In order to estimate the percentage change of data which can be achieved by prediction, it is common to use the R² coefficient in regression analysis [58]. To measure the average intensity of error, both MAE and RMSE were used [59]. Studies have concluded that the model will be more accurate when R² is closer to 1 and MAE and RMSE is closer to 0. These coefficients are determined by the following formulas:

where represents the number of the samples, and were the actual and predicted outputs, respectively, and was the average value of .

4. Results and Discussion

4.1. Effect of Training Data Size

Because the testing set size was selected and fixed to 20% of the total data set and hidden from the training process, the training data set size was chosen by dropping from 60% to 20% of the data set, or meaning the size of the validation set increased from 20% to 60% of the remaining data set. For each size case of the training set, 300 simulations were used, taking into account the random shuffle in the training order. For each size case of the training set, 300 simulations were used, taking into account the random shuffle in the training order. The initialization parameters of the DNN model used in this section are presented in Table 4.

Table 4: Initialization parameters of DNN model.

Parameters
Value and description

Number of hidden layers
2

Neurons structure
(25, 25)

Use early stopping
True

Activation function
Tanh

Cost function
MSE

Data used
Training/validation/testing data set

Number of runs (shuffle input data)
300

From Figure 8, it can be seen that training performance progressively increased with decreasing in the training set. The average R² value was increased from 0.952 to 0.991 when the training set size was decreased from 60% to 20%. On the other side, the predictions showed less accurate results on the validation set and testing set as the training set size decreases. The average R² value was decreased from 0.889 to 0.602 on the validation set, while the average R² value dropped from 0.919 to 0.570 on the testing set. In addition, the standard deviation from the DNN models was increased with decreasing in training set size. To be more specific, the standard deviation was increased from 0.0277 to 0.351 on the testing set. The above results show that the performance of the model got better and became more stable with the most significant training set size case. Aiming to increase the prediction accuracy of the model, the training set size of 60% was selected in this study.

Figure 8: Sensitivity analysis of the model using different training set sizes with DNN model.

4.2. DNN Structure Optimization

The evolutionary results of PSO-DNN models are evaluated in this section. The initialization parameters of PSO-DNN used in this study are given in Table 5. Figure 9 illustrates the evolution of the PSO-DNN model through 30 generations with the number of hidden layers set to values: 2, 4, 6, 8, and 10. A summary of the best predictability of the models is presented in Table 6.

Table 5: PSO-DNN initialization parameters.

Parameters
Explanation
Value and description

N_swarm
Swarm size
50

max_iteration
Number of iterations
30

i
Initial weight
0.5

_damp
Initial weight damping
0.9

C₁
Personal learning coefficient
0.4

C₂
Global learning coefficient
0.8

VarMin
Number of min neurons
2

VarMax
Number of max neurons
100

N_hiddens
Number of hidden layers
2, 4, 6, 8, 10

Vmax
Velocity max
0.05  (VarMax − VarMin)

Vmin
Velocity min
−Vmax

Fitness function

DNN

Activation function

Tanh

Early stopping
Handling overfitting
True

Cost function

R², MAE, RMSE

Data used

Training/validation dataset

(a)

(b)

(c)

Figure 9: Neuron structures tuning using the PSO-DNN model with errors criteria: (a) RMSE; (b) MAE; (c) R².

Table 6: Summary of best prediction capability of PSO-DNN models.

Criteria
Number of hidden layers
Best criteria value
Rank
Best neuron structure

R²
2
0.930
4
(28, 32)

4
0.926
2
(20, 36, 35, 29)

6
0.927
5
(17, 44, 52, 52, 38, 28)

8
0.931
3
(15, 54, 77, 37, 15, 78, 12, 19)

10
0.938
1
(8, 49, 90, 32, 19, 77, 36, 22, 89, 29)

MAE (o)
2
1.465
4
(9, 19)

4
1.426
2
(24, 53, 62, 21)

6
1.444
3
(9, 76, 27, 13, 56, 77)

8
1.528
5
(32, 25, 51, 74, 36, 35, 22, 76)

10
1.395
1
(15, 54, 77, 37, 15, 78, 12, 19, 65, 46)

RMSE (o)
2
1.925
5
(7, 22)

4
1.955
4
(23, 36, 34, 41)

6
1.921
3
(9, 61, 31, 23, 41, 70)

8
1.839
1
(17, 55, 47, 54, 66, 39, 41, 28)

10
1.871
2
(15, 54, 77, 37, 15, 78, 12, 19, 65, 46)

Bold values represent the best model for each performance indicator.

It can be seen that the DNN model with 10 hidden layers has evolved to bring about better performance; the best generation yielded a correlation of R² = 0.938, MAE = 1.395, and RMSE = 1.846. The model with the next best performance is the DNN model with 8 hidden layers which produce accurate intermediate precision (R² = 0.931, MAE = 1.528, and RMSE = 1.839). The DNN model with 10 hidden layer gives the best performance for R² and MAE criteria, while the model with 8 hidden layers gives the best result for the RMSE cost function. However, the best neuron structure for each cost function is different and is also shown in Table 6. The evolutions of number of neurons in 10 hidden layers and 8 hidden layers of the best DNN model referring to R², MAE, and RMSE are presented in Figure 10. It means that the evolution of the model has found three good results; choosing the best model will be done in the next section.

(a)

(b)

(c)

Figure 10: Evolution of neurons in each hidden layer using PSO-DNN models: (a) 10 hidden layers, R²; (b) 10 hidden layers, MAE; (c) 8 hidden layers, RMSE (HD: hidden layers).

4.3. Predictive Capability of the Models

In this section, the three best models, 10 hidden layers, R² (10-HD-R²), 10 hidden layers, MAE (10-HD-MAE), and 8 hidden layers, RMSE (8-HD-RMSE) were chosen for comparison with each other. From the point of view of statistical probability, the random factors in the input variables should be considered. Therefore, 300 simulations take into account the randomness of the input variable sequence number. The data used in this section include training, validation, and testing data sets. The criteria for selecting the best model are based on the results of the testing data set. The results are shown in Figure 11 and Tables 7–9 .

(a)

(b)

(c)

Figure 11: Performance of PSO-DNN models using criteria: R² (a), MAE (b), and RMSE (c).

Table 7: Summary of the 300 simulations using R² criteria.

Model
Data set
Average
Min
Max
SD

10-HD-R²
Training
0.946
0.915
0.970
0.0083

Validation
0.895
0.721
0.939
0.0377

Testing
0.935
0.861
0.966
0.0138

10-HD-MAE
Training
0.900
0.586
0.953
0.0477

Validation
0.851
0.436
0.930
0.0657

Testing
0.889
0.549
0.939
0.0453

8-HD-RMSE
Training
0.935
0.826
0.95
0.0156

Validation
0.901
0.622
0.942
0.0343

Testing
0.881
0.795
0.927
0.0186

Table 8: Summary of the 300 simulations using MAE criteria.

Model
Data set
Average (o)
Min (o)
Max (o)
SD

10-HD-R²
Training
1.188
0.883
1.423
0.0899

Validation
1.721
1.392
2.509
0.2345

Testing
1.340
1.036
1.715
0.1339

10-HD-MAE
Training
1.636
1.050
2.560
0.3031

Validation
2.105
1.473
3.455
0.3411

Testing
1.823
1.243
2.870
0.271

8-HD-RMSE
Training
1.268
1.101
2.107
0.1534

Validation
1.706
1.267
3.444
0.2564

Testing
1.523
1.238
2.338
0.1923

Table 9: Summary of the 300 simulations using RMSE criteria.

Model
Data set
Average (o)
Min (o)
Max (o)
SD

10-HD-R²
Training
1.566
1.260
1.848
0.1088

Validation
2.274
1.782
3.498
0.3395

Testing
1.77
1.406
2.592
0.1835

10-HD-MAE
Training
2.059
1.421
3.128
0.3711

Validation
2.668
1.876
5.095
0.473

Testing
2.255
1.578
3.310
0.2988

8-HD-RMSE
Training
1.722
1.465
3.598
0.2129

Validation
2.22
1.676
4.867
0.3861

Testing
2.019
1.613
3.882
0.3264

The results showed that the predicted results on the training and validation sets gave good accuracy with satisfactory average accuracy R² ranging from 0.851 to 0.946, MAE ranging from 1.188 to 2.105, and RMSE varying from 1.566 to 2.668, in which it can be seen that, among three good models, the 10-HD-R² gives the best prediction, which yielded an average correlation of R² = 0.935, MAE = 1.34, and RMSE = 1.77 on the testing data set. The next best model is 8-HD-RMSE, which produces accurate intermediate precision (R² = 0.881, MAE = 1.523, and RMSE = 2.019). Moreover, the standard deviation of the 10-HD-R² model is better than that of the 8-HD-RMSE, where SD = 0.0138, 0.1339, and 0.1835 for R², MAE, and RMSE criteria compared to SD = 0.0186, 0.1923, and 0.3264 of two models, respectively.

Table 10 shows the comparison between the DNN model and the PSO-DNN model. The DNN model received initialized parameters as shown in Table 11. The best 10-HD-R² PSO-DNN model was included for comparison with the proof of the effectiveness of the algorithm. The average error criteria of 300 simulations were compared to confirm the model’s performance. It can be seen that the 10-HD-R² PSO-DNN model outperformed the DNN model, which improved the average correlation of R² by 1.83%, MAE by 5.94%, and RMSE by 8.58%.

Table 10: Comparison between the DNN model and the PSO-DNN model.

Criteria
DNN model
PSO-DNN model
Performance improvement (%)

R²
0.918
0.935
1.83

MAE (o)
1.425
1.340
5.94

RMSE (o)
1.936
1.770
8.58

Table 11: Comparison with previous studies.

Author
Model
Parameter
Number of samples
R
R²
RMSE (o)
MAE (o)

Khanlari et al. [60]
ANN
Internal friction angle
200
0.89
0.792
2.26
1.92

MVR
0.89
0.79
2.385
1.929

Mohammadi et al. [61]
MLP
Internal friction angle
108
0.902
0.814
9.285
6.137

MLR
0.719
0.517
5.082
3.954

Iyeke et al. [62]
ANN
Internal friction angle
83
0.897
0.805
4.77
4.34

Present study
DNN
Internal friction angle
245
0.958
0.918
1.936
1.425

PSO-DNN
0.967
0.935
1.77
1.34

Table 11 presents some research results on ML applications in determining the internal friction angle of soil. The results of the present study, as well as other studies, show the effectiveness of the ML technique in predicting the internal friction angle of the soil with expected R² from 0.79 to 0.935 on the testing data set. However, due to the different data sets used and the input variables, it makes no sense to compare these results. A project using different data sets as well as various input variables is needed to provide a general model for the prediction of soil shear parameters.

Figure 12 shows a visual comparison of test results and predictions based on the friction angle of soil of the DNN and PSO-DNN models. The performances of two ML models have been tested on all three data sets: training, validation, and testing.

(a)

(b)

(c)

(d)

(e)

(f)

Figure 12: Measured and predicted values of friction angle of soil using the DNN models: PSO-DNN model for (a) training, (c) validation, and (e) testing data set; DNN model for (b) training, (d) validation, and (f) testing data set.

4.4. Sensitivity Analysis

In this section, a global sensitivity analysis was conducted to evaluate the importance of input parameters for the model using Monte Carlo methods [63]. It is an effective way to investigate a relationship between input and output. The input data set taken from Saltelli’s sampling scheme was used to develop the DNN model [64]. The global sensitivity index is determined by the following formulas:

where Var(Y) denotes the total variance of the model output; d denotes the number of input features; Vari denotes the model output variance in response to variation of the ith input variable; Varij denotes the model output variance in response to the simultaneous variation of the ith and the jth input; STi denotes the total sensitivity index.

The result of the total sensitivity analysis was shown in Figure 13. It can be seen that, among the 7 input variables used to predict the friction angle of soil, the elevation of the soil layer bottom (Z₂) was the most important feature, which achieved an average sensitivity index score of 0.625. The topsoil layer elevation (Z₁) was the second important variable, confirmed by an average sensitivity index score of 0.603. From the point of view of soil mechanics, the deeper the soil is, the more compact it is, so the depth of the soil sample plays an important role in detecting the friction angle of the soil when its density, state, and many other mechanical properties change according to the depth. The variables H, G, and S were ranked as the third to the fifth important predictors, with an average sensitivity index ranging from 0.203 to 0.05. Other predictor variables included in the model (e, G) had lower than 0.01 in the sensitivity index, indicating that they had not affected the output prediction. It is important to note that the sensitivity analysis in this section is only relevant for the input data set itself and cannot be confirmed with other data sets.

Figure 13: Sensitivity analysis using the 10-HD-R² model.

5. Conclusions

In this study, a PSO-DNN hybrid model, which can evolve itself to find out the best models, was developed to predict the friction angle of soil. The hybrid model evolved on its own and found three of the best among the survey models. A database containing 245 soil samples from geological boreholes was used to develop and evaluate the three proposed DNN models: 10-HD-R², 10-HD-MAE, and 8-HD-RMSE.

The results show that the models’ performances improved and stabilized from R² = 0.602 to 0.889 on the validation set with the increase in training data set size from 20% to 60%. Research results show that PSO-DLNN obtains the best results with the number of hidden layers from 8 to 10. The optimal number of neurons in each hidden layer is not the same and is distributed intricately in the hidden layer. It recommends that a DLNN model with 8 to 10 hidden layers might be optimal for the problem related to predicting the friction angle of soil. However, it is advisable to select the number of neurons in each hidden layer through evolutionary methods to bring about high efficiency to the DLNN model. The results also showed that, on the training and validation data sets, all three best models, 10-HD-R², 10-HD-MAE, and 8-HD-RMSE, have good predict results, in which the leading is the 10-HD-R² model on the testing data set; the 10-HD-R²-DNN model still gave the best results and outperformed the other two models. Prediction results of 300 simulations show that the 10-HD-R²-DNN model has a smaller standard deviation, indicating that the model is more stable than the other two models.

In addition, the sensitivity analysis using the Monte Carlo method was carried out to evaluate the importance of input features in the model study. The results show that the two inputs related to the depth of the soil layers (Z₂ and Z₁) were considered to be the most important parameters for predicting soil friction angle.

Data Availability

The data used in the manuscript are available in the supplementary materials. Resharing of data with other researchers is completely allowed; however, they need to be cited in the researches.

Conflicts of Interest

The authors declare that they have no conflicts of interest.

Supplementary Materials

Data collected in Ho Chi Minh City, Vietnam, including 245 experimental results for determining the internal friction angle of the soil, together with the input parameters required for machine learning modeling. (Supplementary Materials)

References

M. Ghazavi, M. Hosseini, and M. Mollanouri, “A comparison between angle of repose and friction angle of sand,” 2008.

A. Namdar and G. S. Gopalakrishna, “Analysis of mixed soil model characteristics,” Modern Applied Science, vol. 3, no. 5, 2009.

A. S. Abedi and N. Hataf, “Bearing capacity of strip footings located on grouted soil slopes,” Electronic Journal of Geotechnical Engineering, vol. 19, no. Z4, pp. 16827–16837, 2014.

T. Schanz and P. A. Vermeer, “Angles of friction and dilatancy of sand,” Géotechnique, vol. 46, no. 1, pp. 145–151, 1996.

G. G. Meyerhof, “Some recent research on the bearing capacity of foundations,” Canadian Geotechnical Journal, vol. 1, no. 1, pp. 16–26, 1963.

A. S. Vesic, “Analysis of ultimate loads of shallow foundations,” Journal of the Soil Mechanics and Foundations Division, vol. 99, no. 1, pp. 45–73, 1973.

A. Namdar and M. K. Pelko, “Numerical analysis of soil bearing capacity by changing soil characteristics,” Frattura Ed Integrità Strutturale, vol. 3, no. 10, pp. 38–42, 2009.

K. S. M. SEYED and A. R. RASTI, “Evaluation of soil-structure interaction parameters in static and dynamic response of the retaining wall,” 2017.

P. Jiang and M. S. Zhang, “Real-time monitor method of soil slope stability in seasonal frozen area,” Applied Mechanics and Materials, vol. 303–306, pp. 777–781, 2013.

K. Gelisli, T. Kaya, and A. E. Babacan, “Assessing the factor of safety using an artificial neural network: case studies on landslides in Giresun, Turkey,” Environmental Earth Sciences, vol. 73, no. 12, pp. 8639–8646, 2015.

Y. Mo, X. C. Liu, Z. H. Guo, B. Teng, Z. Y. Zou, and Z. Y. Zhu, “Slip-line field theory's application in soil subgrade,” Applied Mechanics and Materials, vol. 170–173, pp. 283–288, 2012.

H. Yang, J. Wang, and Y. Liu, “A new approach for the slope stability analysis,” Mechanics Research Communications, vol. 28, no. 6, pp. 653–669, 2001.

X. Zhou, J. Li, Z. Liu, and Y. Tang, “Analysis of slope stability with imprecise soil properties using uncertain sets,” Mathematical Problems in Engineering, vol. 13, 2019.

X. Chu, L. Li, and Y. Cheng, ““Risk assessment of slope failure using assumption of maximum area of sliding mass and factor of safety equal to unit,” Advances in Civil Engineering, vol. 23, 2019.

S. Altun, B. Göktepe, and A. Sezer, “Relationships between shape characteristics and shear strength of sands,” Soils and Foundations, vol. 51, no. 5, pp. 857–871, 2011.

P. Vangla and G. M. Latha, “Influence of particle size on the friction and interfacial shear strength of sands of similar morphology,” International Journal of Geosynthetics and Ground Engineering, vol. 1, no. 1, p. 6, 2015.

N. Marachi, J. Duncan, C. Chan, and H. Seed, “Plane-strain testing of sand,” Laboratory Shear Strength of Soil, vol. 23, 1981.

T. Schanz and J. Alabdullah, “Testing unsaturated soil for plane strain conditions: a new double wall biaxial device,” Springer Proceedings in Physics, vol. 23, pp. 169–178, 2016.

K. L. Dev, R. J. Pillai, and R. G. Robinson, “Drained angle of internal friction from direct shear and triaxial compression tests,” International Journal of Geotechnical Engineering, vol. 10, no. 3, pp. 283–287, 2016.

A. L. Kryzhanovskii, Y. S. Vil'gel'm, and S. V. Medvedev, “Determination of the friction angle of soils in triaxial-compression apparatus and shear devices,” Soil Mechanics and Foundation Engineering, vol. 22, no. 3, pp. 105–111, 1985.

Ş. Cioară, A. Stanciu, and I. Lungu, “Comparative study ON determining the internal friction angle for sand,” Soil Mechanics and Foundation Engineering, vol. 12, 2014.

S. Nanda and N. R. Patra, “Determination of soil properties for plane strain condition from the triaxial tests results,” International Journal for Numerical and Analytical Methods in Geomechanics, vol. 39, no. 9, pp. 1014–1026, 2015.

J. C. Santamarina and G. C. Cho, “Determination of critical state parameters in sandy soils - simple procedure,” Geotechnical Testing Journal, vol. 24, no. 2, 2001.

J. H. Choi, S. Dai, J. S. Lin, and Y. Seol, “Multistage triaxial tests on laboratory‐formed methane hydrate‐bearing sediments,” Journal of Geophysical Research: Solid Earth, vol. 123, no. 5, pp. 3347–3357, 2018.

B. R. Murlidhar, M. Ahmed, D. Mavaluru, A. F. Siddiqi, and E. T. Mohamad, “Prediction of rock interlocking by developing two hybrid models based on GA and fuzzy system,” Engineering with Computers, vol. 35, no. 4, pp. 1419–1430, 2019.

H. Mujtaba, K. Farooq, N. Sivakugan, and B. M. Das, “Evaluation of relative density and friction angle based on SPT-N values,” KSCE Journal of Civil Engineering, vol. 22, no. 2, pp. 572–581, 2018.

H. Hettiarachchi and T. Brown, “Use of SPT blow counts to estimate shear strength properties of soils: energy balance approach,” Journal of Geotechnical and Geoenvironmental Engineering, vol. 135, no. 6, pp. 830–834, 2009.

M. Hatanaka and A. Uchida, “Empirical correlation between penetration resistance and internal friction angle of sandy soils,” Soils and Foundations, vol. 36, no. 4, pp. 1–9, 1996.

R. Salgado, P. Bandini, and A. Karim, “Shear strength and stiffness of silty sand,” Journal of Geotechnical and Geoenvironmental Engineering, vol. 126, no. 5, pp. 451–462, 2000.

P. K. Robertson and R. G. Campanella, “Interpretation of cone penetration tests. Part I: Sand,” Canadian Geotechnical Journal, vol. 20, no. 4, pp. 718–733, 1983.

Y. Yoshida and K. Kokusho, “A proposal on application of penetration tests on gravelly soils,” Denryoku Chuo Kenkyusho Hokoku, vol. 1, no. 1–4, 1988.

R. M.-M. Y. Rad2-Sina, S. Haghshenas, M. Z. Naghadehi, and R. Branch, “Risk management of a mechanized tunneling project through assessment of geological units using an evolutionary method,” 1988.

X. Liangfu and C. Jiangtao, “Predicting the mechanical parameters of holocene alluvial layers in the yangtze river delta using BP neural network based on cone penetration test,” 1988.

S. K. Das and P. K. Basudhar, “Prediction of residual friction angle of clays using artificial neural network,” Engineering Geology, vol. 100, no. 3-4, pp. 142–145, 2008.

S. A. Al-Hamed, M. F. Wahby, and A. M. Aboukarima, “Artificial neural network for soil cohesion and soil internal friction angle prediction from soil physical properties data,” 2014.

B. T. Pham, C. Qi, L. S. Ho et al., “A novel hybrid soft computing model using random forest and particle swarm optimization for estimation of undrained shear strength of soil,” Sustainability, vol. 12, no. 6, p. 2218, 2020.

D. T. Bui, P. Tsangaratos, V.-T. Nguyen, N. V. Liem, and P. T. Trinh, “Comparing the prediction performance of a Deep Learning Neural Network model with conventional machine learning models in landslide susceptibility assessment,” Catena, vol. 188, 2020.

D. T. Bui, “A novel deep learning neural network approach for predicting flash flood susceptibility: a case study at a high frequency tropical storm area,” Science of the Total Environment, vol. 701, p. 134413, 2020.

C. D. Storie and C. J. Henry, “Deep learning neural networks for land use land cover mapping,” 2018.

R. Zemouri, N. Omri, F. Fnaiech, N. Zerhouni, and N. Fnaiech, “A new growing pruning deep learning neural network algorithm (GP-DLNN),” 2019.

F. Vandenbergh and A. Engelbrecht, “A study of particle swarm optimization particle trajectories,” Information Sciences, vol. 176, no. 8, pp. 937–971, 2006.

M. Khajehzadeh, M. R. Taha, A. El-Shafie, and M. Eslami, “Modified particle swarm optimization for optimum design of spread footing and retaining wall,” Journal of Zhejiang University-Science A, vol. 12, no. 6, pp. 415–427, 2011.

J. Meier, W. Schaedler, L. Borgatti, A. Corsini, and T. Schanz, “Inverse parameter identification technique using PSO algorithm applied to geotechnical modeling,” Journal of Artificial Evolution and Applications, vol. 2008, 2008.

P. V. Lade and C.-T. Liu, “Experimental study of drained creep behavior of sand,” Journal of Engineering Mechanics, vol. 124, no. 8, pp. 912–920, 1998.

R. Saito, H. Fukuoka, and K. Sassa, “Experimental study on the rate effect on the shear strength,” 1998.

T. Aydogmus and H. Klapperich, “Three-dimensional numerical parametric analysis of soil-geosynthetic-interaction,” Journal of Soil Mechanics & Foundations Divisions, vol. 171, p. 8, 2008.

G. G. Meyerhof, “Shallow foundations,” Journal of Soil Mechanics & Foundations Divisions, vol. 91, 2009.

B. Sharma and A. Sridharan, “Liquid and plastic limits of clays by cone method,” International Journal of Geo-Engineering, vol. 9, no. 1, p. 22, 2018.

S. P. K. Kodicherla and D. K. Nandyala, “Use of CPT and DCP based correlations in characterization of subgrade of a highway in Southern Ethiopia Region,” International Journal of Geo-Engineering, vol. 7, no. 1, p. 11, 2016.

N. Q. A. M. Yusof and H. Zabidi, “Reliability of using standard penetration test (SPT) in predicting properties of soil,” Journal of Physics: Conference Series, vol. 1082, 2018.

M. Farzi and A. Saligheh Zadeh, “Determination of relationship between standard penetration test and geotechnical parameters in Ahwaz soil,” Geomechanics and Geoengineering, vol. 23, pp. 1–13, 2019.

J. Kennedy and R. Eberhart, “Particle swarm optimization,” Proceedings of ICNN’95-International Conference on Neural Networks, vol. 4, pp. 1942–1948, 1995.

D. Tien Bui, V.-H. Nhu, and N.-D. Hoang, “Prediction of soil compression coefficient for urban housing project using novel integration machine learning approach of swarm intelligence and Multi-layer Perceptron Neural Network,” Advanced Engineering Informatics, vol. 38, pp. 593–604, 2018.

Y. M. Cheng, L. Li, S.-C. Chi, and W. B. Wei, “Particle swarm optimization algorithm for the location of the critical non-circular failure surface in two-dimensional slope stability analysis,” Computers and Geotechnics, vol. 34, no. 2, pp. 92–103, 2007.

B. G. Lipták, Process Control Instrument Engineer, Elsevier, London, UK, 1995.

C. Van Der Malsburg and F. Rosenblatt, Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms, Elsevier, Berlin, Heidelberg, 1986.

T. A. Pham, V. Q. Tran, H.-L. T. Vu, and H.-B. Ly, “Design deep neural network architecture using a genetic algorithm for estimation of pile bearing capacity,” PLoS One, vol. 15, no. 12, 2020.

S. Menard, “Coefficients of determination for multiple logistic regression analysis,” The American Statistician, vol. 54, no. 1, pp. 17–24, 2000.

T. Chai and R. R. Draxler, “Root mean square error (RMSE) or mean absolute error (MAE)?-arguments against avoiding RMSE in the literature,” Geoscientific Model Development, vol. 7, no. 3, pp. 1247–1250, 2014.

G. R. Khanlari, M. Heidari, A. A. Momeni, and Y. Abdilor, “Prediction of shear strength parameters of soils using artificial neural networks and multivariate regression methods,” Engineering Geology, vol. 131-132, pp. 11–18, 2012.

M. Mohammadi, S. M. Fatemi Aghda, M. Talkhablou, and A. Cheshomi, “Prediction of the shear strength parameters from easily-available soil properties by means of multivariate regression and artificial neural network methods,” Geomechanics and Geoengineering, vol. 0, no. 1, pp. 1–13, 2020.

S. Iyeke, E. Eze, J. Ehiorobo, and S. Osuji, “Estimation OF shear strength parameters OF lateritic soils using artificial neural network,” Nigerian Journal of Technology, vol. 35, no. 2, p. 260, 2016.

I. M. Sobol′, “Global sensitivity indices for nonlinear mathematical models and their Monte Carlo estimates,” Mathematics and Computers in Simulation, vol. 55, no. 1, pp. 271–280, 2001.

A. Saltelli, “Making best use of model evaluations to compute sensitivity indices,” Computer Physics Communications, vol. 145, no. 2, pp. 280–297, 2002.