Estimation of State of Charge of Lithium-Ion Batteries Based on Wide and Deep Neural Network Model

. It is important to accurately estimate the SOC to ensure that the lithium-ion battery is within a safe working range, prevent over-charging and over-discharging, and ultimately improve battery life. However, SOC is an internal state of the battery and cannot be directly measured. This paper proposes a SOC estimation method based on the wide and deep neural network model, which combines the linear regression (LR) model and the backpropagation neural network (BPNN) model. This article uses the dataset provided by the Advanced Energy Storage and Applications (AESA) group to verify the performance of the model. The performance of the proposed model is compared with the common BPNN model in terms of root mean square error (RMSE), average absolute proportional error (MAPE), and SOC estimation error. The validation results prove that the eﬀect of the proposed model in estimating SOC is better than that of the ordinary BPNN model. Compared with the BPNN model, the RMSE values of the SOC predicted value of the wide and deep model in the charging and discharging stages were reduced by 10.2% and 15.4%, respectively. Experimental results show that the maximum SOC estimation error of the model in predicting the SOC during charging and discharging is 0.42% and 0.86%, respectively.


Introduction
With the massive use of fossil fuels, environmental pollution and energy shortages have become increasingly serious. Governments are paying more and more attention to the development and utilization of clean energy. In recent years, electric vehicles have received great attention from the government and enterprises [1]. Different types of electric vehicles have emerged in the market, such as pure electric vehicles (BEVs), hybrid electric vehicles (HEVs), fuel cell vehicles (FCEVs), and so on [2]. Rechargeable batteries are widely used in electric vehicles, such as lead-acid batteries, nickel-cadmium batteries, nickel-hydrogen batteries, and lithium-ion batteries. Among them, lithium-ion batteries are favored by many companies due to their advantages such as long life, low pollution, high power, and fast charging, and its market proportion is also constantly increasing [3]. erefore, battery management technology has attracted much attention in recent years. In [4], the authors proposed an enabling state of health (SOH) estimation scheme based on the ICA method for real-world EVs. is is realized by combining an equivalent IC-value calculation for battery packs with cell-level battery tests while taking cell inconsistency into consideration. In [5], the authors proposed a multistage alternative current (AC) strategy for internally hearting lithium-ion batteries. In [6], a data-driven method was proposed for battery charging capacity diagnosis based on massive real-world EV operating data. e estimation of state of charge (SOC) of lithium-ion batteries is also a key part of battery management technology. Lithium-ion batteries will age, that is, their capacity will gradually decay as the number of uses increases. ere are many reasons for the accelerated decay rate of lithiumion battery capacity, including excessive temperature, overcharging, over-discharging, and not using it for a long time after being fully charged [7]. To prevent over-charging and over-discharging, there must be an accurate estimation of the SOC of the lithium-ion battery. e state of charge (SOC) is a relative measure of the energy stored in a battery, defined as the ratio of the amount of charge that can be extracted from a battery cell at a specific point in time to the rated capacity [8]. e calculation formula of SOC can be expressed as where Q c is the remaining capacity of the battery and C N is the rated capacity. It is important to accurately estimate the SOC to ensure that the lithium-ion battery is within a safe working range, prevent over-charging and over-discharging, and ultimately improve battery life. Because the capacity that the battery can release is affected by many factors, including discharge rate, discharge current, battery internal temperature, self-discharge, charge and discharge cycles, battery aging, and so on, the SOC must be related to these factors. ere are many traditional methods for SOC estimation, including the coulomb counting method [9,10], open circuit voltage method [11], Kalman filtering method, and so on [12,13]. e coulomb counting method is the most commonly used SOC estimation method. If initial state is SOC 0 , then the current SOC is where I is the battery current and η is the efficiency of charging and discharging. e shortcomings of this method are as follows: inaccurate current measurement will cause SOC calculation errors, which will accumulate for a long time, and the error will become larger and larger; the battery charging and discharging efficiency must be considered; and the error will be larger under the condition of high temperature and severe current fluctuations. e open circuit voltage of the battery is close to the electromotive force of the battery in value. ere is a relatively fixed functional relationship between the open circuit voltage and the SOC [14], so that the SOC can be estimated based on the open circuit voltage. e obvious disadvantage is that the battery needs to be left to stand for a long time to achieve voltage stability [11]. It takes several hours to restore the battery state from work to stability, which causes difficulties in measurement. How to determine the resting time is also a problem, so the method is only suitable for the parking state of electric vehicles. e open circuit voltage method is effective in estimating SOC in the initial and final stages of charging and is often used in combination with the coulomb counting method. e core idea of Kalman filter theory is to make an optimal estimation of the state of the dynamic system in the sense of minimum variance. is method is suitable for all kinds of batteries. Compared with other methods, it is especially suitable for the estimation of the battery SOC of hybrid electric vehicles with severe current fluctuations. It not only gives the estimated value of SOC but also gives the estimation error of SOC.
With the rapid development of machine learning technology in recent years, more and more researchers have begun trying to use data-driven methods to make predictions about SOC. ese methods can automatically learn network parameters through intelligent algorithms and obtain the relationship between battery parameters and SOC from them [15,16]. Machine learning methods commonly used for SOC estimation include neural networks [17], support vector machine (SVM) [18], extreme learning machine (ELM) [19], and so on. Recent neural network structures for SOC estimation mainly include wavelet neural network (WNN) [20,21], backpropagation neural network (BPNN) [22], radial basis function neural network (RBFNN) [23], and so on. In [24], the authors combined adaptive wavelet neural networks with a discrete wavelet transform, proposing a new hybrid wavelet neural network model based on the Levenberg-Marquardt algorithm to estimate the SOC. In [25], an improved BPNN-based SOC estimation method for lithiumion batteries was proposed. e method used two algorithms: principal component analysis and particle group optimization, to improve the accuracy and robustness of the model.
Neural network has strong expression ability and generalization ability but lacks "memory ability." "Memory ability" can be understood as the ability of the model to directly learn and use the "co-occurrence frequency" of items or features in historical data. e linear model has strong memory ability. is article uses the wide and deep model to estimate the SOC. e wide and deep model is a hybrid model composed of a singlelayer wide part and a multilayer deep part [26]. Among them, the wide part is a single-layer linear model whose main function is to make the model have the "memory ability," and the deep part is the BPNN model whose main function is to make the model have generalization ability. It is this structural feature that makes the model have the advantages of both linear model and deep neural network. erefore, the wide and deep model can swiftly learn and memorize a large number of historical behavior characteristics while also having great expressive ability. e rest of this article is organized as follows. In Section 2, the principle and basic structure of the wide and deep model are introduced in detail, and how to use it for SOC prediction is also given. In Section 3, the method of obtaining training data and the original data analysis are introduced. Section 4 gives the experimental results and analysis. Section 5 draws the conclusion.

Wide and Deep Model
e wide and deep model connects the wide part of the single input layer with the deep part, which is composed of the embedding layer and multiple hidden layers, and then inputs them to the final output layer. e wide part is good at handling a large number of sparse category features. e deep part uses the strong expressive ability of the neural network to mine the data patterns hidden behind the features. Finally, using the linear regression model, the output layer combines the wide part and the deep part to form a unified model, as shown in Figure 1.

Wide Part.
e wide part is a generalized linear model, which can be expressed as follows: where y is the predictive value; w T � [w 1 , w 2 , . . . , w n ] is the weight of features; x � [x 1 , x 2 , . . . , x n ] T is the feature vector; and b is the bias.
2 Mathematical Problems in Engineering e features utilized in the wide part are usually original features or new features made by applying some simple transformations to the original features. Cross-product transformation is the most common transformation method employed, and it is typically applied to category features. Table 1 shows the data of a group of users watching videos. e original data have only two features: user id: u 1 , u 2 , u 3 andvideo id: f 1 , f 2 . After one-hot encoding, the dimensions of these two features are 3 and 2, respectively, and the corresponding cross-product features of these two features are user id × video id: , with a dimension of 6. e nonlinear features obtained by the cross-product transformation can capture the high-order correlation between the original features. If the original dense feature is to be subjected to cross-product transformation, it needs to be discretized. e advantage of the wide model with cross-product features is that the model has high interpretability and the feature importance is easy to analyse. ere are three main disadvantages. One is that a lot of manual design is required. e other is that if the original category feature dimension is too high, then the feature dimension obtained through cross-product transformation will be very large, which will cause over-fitting problems. e solution is to construct more coarse-grained cross features. e third is that when the number of original features n is too large, in order to obtain high-order correlation between features, the number of cross-product features that need to be constructed is O(n 2 ), which is basically impossible. At this time, a deep neural network needs to be used.

Deep Part.
e deep part is a backpropagation neural network, which consists of three components: input layer, hidden layer, and output layer. e input features generally include sparse and dense features. e dense features can be input directly. e sparse feature must be encoded into a vector form before input. A simple one-hot encoding method can be employed when the sparse feature dimension is low. When the sparse feature dimension is too high, the use of onehot encoding will cause the problem of dimensional explosion. At this time, the sparse feature can be trained into a lowdimensional word vector.
After the feature is input, it will enter the hidden layer, and each neuron in the hidden layer will perform the following operations: where W l i , b l i are the weight and bias of the i-th neuron in the l-th hidden layer; x l is the activation value of the l-th hidden layer, and its dimension is equal to the number of neurons in the l-th layer; x l+1 i is the i-th activation value of the l + 1-th hidden layer; and f is the activation function. is article uses the Relu function as the activation function, and its function expression is as follows: e deep part can make up for the defects of the wide part. e deep part does not require much manual participation, but it can have better generalization of the cross features that the wide part cannot construct.

Joint Part.
e joint part uses the log-weighted sum of the output of the wide layer and the output of the deep layer as the final predicted value. e calculation formula is as follows: where σ is the sigmoid function; w T wide is the weight of the wide layer; w T deep is the weight used for the final activation value of the deep layer; and x lf deep is the activation value of the last hidden layer. e model selects logistic loss as the loss function, and the loss is calculated as follows: where m is the number of training samples; y (i) , y (i) p are the true value and predicted value of the i-th sample, respectively; and Ω is the regularization term. e weights and biases need to be updated according to the loss. Using AdaGrad as the optimizer of the model, the calculation method is as follows: , where η is the learning rate; ε is a very small value to prevent the denominator from being 0; and G t is the square accumulation of the gradient of the parameter.

Estimating SOC with the Wide and Deep Model.
is article uses the wide and deep model to predict the SOC state value during the charging and discharging process of lithium-ion batteries. e features used are current i, voltage v, dv, di, d 2 v, d 2 i, and the battery charge cycles. All features must be normalized before entering the model. In this experiment, the max-min normalization method was used to normalize the value to [0,1]. e formula is as follows: where x i is the original value; x min , x max are the minimum and maximum values of the feature, respectively; and x i ′ is the normalized value. e input of the wide part includes the original 7 features. For the deep part, the 7 original features were first processed into embedding vectors by PCA and then input into the model. e model structure is shown in Figure 2.

Data Sources.
In order to verify the performance of the wide and deep model, this paper uses the lithium-ion battery charge and discharge data of the Mendeley dataset of AESA. ese data are obtained by charging and discharging two lithium-ion batteries with a standard capacity of 27 Ah for 100 times. When charging, the CC-CV protocol is used. First, the battery is charged with a constant current of 27 A until the terminal voltage reaches 4.2 V, and then the terminal voltage is kept at 4.2 V for constant voltage charging until the current drops to 2.7 A to stop charging. When discharging, adopt the CC discharge protocol. Keeping the current at 81 A, stop discharging when the terminal voltage drops to 2.75 V. e above process is repeated 100 times, with an interval of 30 minutes each time, and the ambient temperature is always 40°C. e overall procedure of the lithium-ion charge and discharge experiment is shown in Figure 3.

Changes of the Battery Maximum Charge Capacity.
In practice, the maximum charge capacity of the battery will reduce with the increase in the number of charges. Since the current, voltage, and temperature are strictly controlled during the experiment, over-charging, over-discharging, and the influence of the external environment on the battery capacity are avoided. erefore, the capacity degradation of the battery in this experiment is mainly caused by the internal factors of the battery. In terms of the electrode, repeated charging and discharging reduce the active surface area of the electrode, the structure of the active material changes, and the electrical contact of the active particles becomes worse. In the electrolyte solution, the electrolyte or the decomposition of conductive salt causes its conductivity to decrease, and the decomposition product causes interface passivation. e change in the charge capacity of the two batteries in this experiment is shown in Figure 4(a). Figure 4(b) shows the difference in capacity between the two batteries. It can be seen that the largest difference in capacity between them is only 0.122 Ah. e change in battery capacity is almost the same.

e Difference between Charge Capacity and Discharge
Capacity. During each round of charging and discharging, the discharge capacity of the lithium-ion battery is often smaller than the charging capacity, as shown in Figures 5 and  6. is is mainly caused by two reasons. One is that during the first few rounds of charging, a solid electrolyte interface (SEI) is formed on the surface of the negative electrode, which consumes lithium ions from the positive electrode. ese lithium ions no longer participate in the subsequent     Figure 7 shows the change in the terminal voltage with the charging time in different rounds. It can be clearly seen from Figure 7(c) that as the number of charge and discharge increases, the terminal voltage will reach 4.2 V faster during charging. Figure 8 shows the changes in current and SOC over time after entering constant voltage charging.

Performance Indicators.
is article uses two statistical errors to verify the performance of the wide and deep model: root mean square error (RMSE) and mean absolute percentage error (MAPE). eir calculation formulas are as follows: where soc true , soc estimate are the real SOC and the estimated SOC, respectively, and N is the number of samples. Mathematical Problems in Engineering

Neural Network Parameter Selection.
is paper selects the experimental data of the 50th charge and discharge as the verification set and the rest as the training set. e backpropagation neural network (BPNN) and wide and deep model are used in this experiment to estimate the SOC. e number of training epochs and the number of hidden neurons in each hidden layer are critical parameters that may influence the performance of the neural network on SOC estimation. is paper conducts various experiments to select the best parameters of the two models. e data of battery 1 during charging stage are used to study the influence of these parameters. Firstly, the influence of the number of training epochs on the estimation performance is studied. To be fair, the learning rate and number of hidden neurons are same in the two models. e values of RMSE and MAPE in different training epochs are plotted in  Next, the performance of the two models is tested under different hidden nodes, ranging from 0 to 200 at intervals of 10. e results are plotted in Figure 10. Generally speaking, using more neurons can better fit the training set, but it will lead to over-fitting problems. On the other hand, too few neurons cannot effectively capture the high-order relationship between input and output. From this perspective, combined with the experimental results, 140 hidden nodes are recommended for the wide and deep model and 150 hidden nodes are recommended for the BPNN model.

SOC Estimate Results.
is paper selects the experimental data of the 50th charge and discharge as the verification set and the rest as the training set. Figures 11-14 show the SOC prediction results of battery 1 and battery 2 at the 50th charge and discharge stage. e solid line in Figures 11(a)-14(a) represents the real SOC value, and the dotted line and point line are the SOC values predicted by the wide and deep model and BPNN model, respectively. Figures 11(b)-14(b) show the errors between the corresponding real SOC and the predicted SOC. It can be seen from the figures that the difference between the SOC predicted by the wide and deep model and the real SOC is very small. e maximum prediction errors of battery 1 during the charging and discharging stage are 0.42% and 0.86%, respectively.
e maximum prediction errors of battery 2 during the charging and discharging stage are 0.50% and 0.56%, respectively.

Performance Comparison.
is paper uses the common backpropagation neural network (BPNN) to compare various performance indicators with the wide and deep model. Table 2 shows the prediction performance of these two models. Next, the performance of the SOC prediction result of battery 1 is described in detail. Compared with the BPNN model, the RMSE values of the SOC predicted value of the wide and deep model in the charging and discharging stages are reduced by 10.2% and 15.4%, respectively. e values of MAPE in the charging and discharging stages for the proposed model are improved and reduced by 53.4% and 26.9%, respectively, in comparison to the common BPNN model. In addition, the SOC error interval of the wide and deep model was smaller than that of the BPNN model. e maximum prediction errors of BPNN model during the charging and discharging stage are 0.50% and 1.39%, respectively. e maximum prediction errors of the wide and deep model during the charging and discharging stages are 0.42% and 0.86%, respectively.

Conclusion
is paper proposes a method to predict the SOC state of lithium-ion batteries based on the wide and deep neural network model. is paper uses the experimental data of 100 times of charging and discharging of two lithium-ion batteries as the training and validation data for the model. e features selected were i, v, di, dv, d 2 i, d 2 v, and the number of charging cycles. e wide part of the wide and deep model used a linear regression model, and the input included the original 7 features. e deep part used the BPNN model, and the input was the embedding vector of the 7 original features.
Compared with the common BPNN model, the SOC prediction method proposed in this paper did not consume more time and computing resources, but the accuracy and stability of the prediction were significantly improved. For example, the values of MAPE in the charging and discharging stages for the proposed model are improved and reduced by 53.4% and 26.9%, respectively, in comparison to the common BPNN model. Compared with the BPNN model, the RMSE values of the SOC predicted value of the wide and deep model in the charging and discharging stages are reduced by 10.2% and 15.4%, respectively.
However, the data used in this article were obtained under ideal constant temperature conditions, so the model did not take the influence of temperature into account. In addition, the battery always adopted CC-CV charging and CC discharging protocols, which is rare in real life. In subsequent research, temperature can also be used as a feature, and at the same time, a more realistic charging and discharging protocol can be used to verify the performance of the wide and deep model.

Data Availability
e data used to support the findings of this study can be accessed by visiting https://data.mendeley.com/datasets/ c5dxwn6w92/1.