PINN-CDR: A Neural Network-Based Simulation Tool for Convection-Diffusion-Reaction Systems

. In this paper, a discretization-free approach based on the physics-informed neural network (PINN) is proposed for solving the forward and inverse problems governed by the nonlinear convection-difusion-reaction (CDR) systems. By embedding physical information described by the CDR system in the feedforward neural networks, PINN is trained to approximate the solution of the system without the need of labeled data. Te good performance of PINN in solving the forward problem of the nonlinear CDR systems is verifed by studying the problems of gas-solid adsorption and autocatalytic reacting fow. For CDR systems with diferent P´eclet number, PINN can largely eliminate the numerical difusion and unphysical oscillations in traditional numerical methods caused by high P´eclet number. Meanwhile, the PINN framework is implemented to solve the inverse problem of nonlinear CDR systems and the results show that the unknown parameters can be efectively recognized even with high noisy data. It is concluded that the established PINN algorithm has good accuracy, convergence, and robustness for both the forward and inverse problems of CDR systems.


Introduction
Reacting fow models play an important role in the simulation of many physical and chemical problems, such as the pollutant transport process in water and air [1], heat conduction process in fowing fuids [2], chromatography column in reactors [3], and high-speed eddy current in electromagnetic felds [4]. A reacting fow model is often composed of a group of convection-dominated partial differential equations (PDEs) with nonlinear source terms [5][6][7], which usually accompanies autocatalytic reactions. A typical reacting fow model is the convection-difusionreaction (CDR) system [8], which is one kind of basic PDE with nonlinear source terms of autocatalytic reactions [9]. Te so-called autocatalytic reaction means that through mutation, the autocatalyst will be transformed into another form of substance, and this new substance can also undergo an autocatalytic reaction at the same time, and eventually lead to competition between the new substance and the original autocatalyst [10]. Due to the complexity of autocatalytic reactions, some of the parameters such as kinetic parameters, mutation parameters and convective difusion coefcients are often unavailable. Terefore, the CDR problem can be further divided into the forward and inverse problems. Te forward problem refers to solving the concentration of reactants at various points within a reaction for which all boundary conditions and medium parameters are known, while the inverse problem refers to the recognition of media parameters by limited known data. By accurately solving these reacting fow models, the reaction problems in chemistry, physics, electromagnetism, and fuids can be analyzed, and suitable reaction units can be designed to optimize the process control schemes. Terefore, it is important to develop an accurate and efcient simulation tool for solving both the forward and the inverse problems of CDR systems.
For the forward problem of CDR involved in fuids, most of the traditional methods are numerical ones. Numerical schemes play a key role in the study of reacting fows and a large variety of efcient numerical methods have been developed, including fnite diference [11], fnite volume [12], fnite element [13], and spectral methods [14]. Te core of these numerical methods is to use some discrete structure to reduce the infnite dimensional operators to a fnite dimensional approximation problem, that is, to divide a large space-time region into multiple, small, and simple regions that are easily processed by computers. Tey are used to numerically solve diferent types of PDEs for large variety of static and dynamic problems. However, these numerical methods are often computationally cumbersome, especially for the problems with moving steep front and complex geometries. Moreover, mesh generation usually incurs a huge burden.
With the explosive growth of computing resources over the past decade, deep learning [15,16], especially, deep neural network (DNN) [17,18], has undergone revolutionary development. It is increasingly used to solve fundamental PDEs in physics and chemistry problems [19][20][21][22], with the help of the general approximation theorem of neural networks and their powerful characterization capabilities, i.e., excellent nonlinear approximation of the model by the combination of multiple hidden layers and nonlinear activation functions. Nevertheless, deep learning introduces new uncertainties and other drawbacks to reacting fow problems. For example, generating an accurate alternative model of a complex physical system usually requires an extremely large sample of data, which is often prohibitively expensive or infeasible to be obtained from measurements or simulations in reality.
In recent years, a DNN framework named physicsinformed neural network (PINN) [23,24] was developed. PINN does not require manually labeled training data. No validation and testing dataset are needed. Tis largely difers from other deep neural networks. Making the full-use of physical information as prior knowledge, PINN is trained with few or even no labeled data as surrogate models for accurate solution of PDEs [25]. Diferent from traditional mesh-based discretization methods, time and space derivatives in the PINN method are evaluated using automatic diferentiation [26] of the DNN that does not involve any numerical discretization. Ten, the DNN coefcients are computed by minimizing the loss function that is the sum of the residuals of both the PDEs and initial and boundary conditions [24]. In addition, the PINN solution defnes a function over the continuous domain, rather than a discrete solution on a grid as in traditional methods. Only initial and boundary conditions are needed to train the PINN to accurately approximate the solution of the equations.
For the inverse problem of CDR, systematic identifcation and thus reconstruction of source features from sparse data are very important. However, the inverse problem is always a high challenging topic, for which many difculties exist, such as the inherent ill-posedness, data uncertainties, and sparse and noisy observation data. To solve the inverse problem, some methods have been developed such as genetic algorithms (GAs) [27], simulated annealing (SA) [28], adaptive simulated annealing (ASA) [29], artifcial neural networks (ANNs) [30], and harmony search (HS) [31]. Tese methods are signifcantly afected by the noise in the observed data. Some nonclassical optimization algorithms, namely, the population-based ones (e.g., GA), also require a great number of evaluations of the objective function, which is computationally expensive [32]. However, PINN not only solves the forward problem according to the governing equation and the initial and boundary conditions, but also solves the inverse problem according to the sparse observed data. It learns the unknown parameters of the system from a small amount of given data and has strong robustness to noise [33,34], which can be a new way for solving the inverse problem of CDR.
Due to the excellent capability of neural networks in describing complex relationship between inputs and outputs, PINN creates a new path to solve the forward and inverse problems involving nonlinear PDEs [25]. Noisy, sparse, and multifdelity data sets are easily handled by PINN. Nowadays, many problems difcult for traditional numerical methods are solved by using the PINN-based methods [35][36][37]. PINN has been successfully used for solving PDEs or complex PDE-based problems in various domains, such as fuid mechanics [38,39], medical diagnosis [40,41] and materialogy [42]. PINN has been applied to single reactant CDR problems with good results [43]. However, there are no researches on the application of PINN to CDR systems with multiple coupled reactants. Te objective of this paper is to solve the forward and inverse problems for multireactant CDR systems. Te PINN is applied to the gas-solid adsorption problem and the autocatalytic reacting fow problem in a tubular reactor. In the autocatalytic reacting fow problem, the use of sin instead of the standard activation functions such as tanh improves the learning ability of the network for high-frequency signals. In addition, the arithmetic examples examine the computational accuracy and stability of the algorithm for the inverse problem with diferent amounts of training data and different levels of noise. Te results show that the PINN algorithm developed in this paper is a new, simple, and efective simulation tool for solving the forward and inverse problems of CDR systems.
Te rest of the paper is organized as follows: In Section 2, the gas-solid adsorption model and autocatalytic reacting fow model are presented. In Section 3, the PINN method for 2 International Journal of Intelligent Systems solving the forward and inverse problems is introduced, and in Section 4, PINN is tested by two CDR systems, including both the forward and inverse problems. Conclusions are given in Section 5.

Reactor Models with Multicomponent Reactant
In this section, two kinds of multicomponent reactant models are introduced, including the gas-solid adsorption model and the autocatalytic reacting fow model.

Gas-Solid Adsorption
Model. Te gas-solid adsorption column without the difusion efect is described by one PDE for fow transport, one diferential equation for mass transfer and one algebraic equation for equilibrium state is [44] as follows: where gas concentration (C G ), solid concentration (C S ), gas-solid interface concentration (C I ), void fraction (ϵ), superfcial gas velocity (v), speed mass transfer coefcient (k), and speed equilibrium constant (K) are denoted. Indices t and x are used for temporal and spatial derivatives, i.e., C t and C x , respectively.

Autocatalytic Reacting Flow Model.
Te chemical reactor is of the tubular type, where chemical species fow from the inlet to the outlet in one pass. Te chemical reaction model includes a cubic autocatalytic reaction in which the autocatalyst is assumed to undergo a mutational process that produces another form, and it can also undergo an autocatalytic reaction and thus compete with the original autocatalyst [45]. Te model captures the fundamental steps encountered in many technically important biochemical and pharmaceutical applications, such as the birth-death process of bacteria and the interaction of drugs with some other biological agents or cells. Te autocatalytic reaction consists of three reagents (substrate A, autocatalyst B, and mutant C) and is carried out according to the following reaction scheme [46]: Specifcally, both k 1 and k 2 represent the reacting rate constants, α is the mutation constant, and β is the mutation efciency. For simplicity, we assume that the fow rate along the reactor is constant. Ten, the transport equation describing the three reactants in dimensionless form can be expressed as follows [45]: where In the above equations, U i (i � 1, 2, 3) is the dimensionless concentration of the reactants A, B, and C, u f is the substrate concentration, X is the dimensionless reactor length, T is the dimensionless time, L represents the length of the tubular reactor, D i (i � 1, 2, 3) is the dimensionless difusion parameter, v is the dimensionless convection velocity, and c is a dimensionless kinetic parameter.

Physics-Informed Neural Network (PINN)
Te PINN is a machine learning framework based on DNN. It leverages the capabilities of DNN as universal function approximators. However, diferent from traditional deep learning algorithms, PINN restricts the set of acceptable solutions by enforcing the validity of PDE models governing the actual physics of the problem. Tis is achieved within a fully connected feedforward neural network architecture leveraging automatic diferentiation techniques available in the TensorFlow learning package [47]. Te basic idea of the PINN algorithm is to embed the governing equations of physical prior information (such as conservation quantity, invariance, and symmetry) into the loss function International Journal of Intelligent Systems corresponding to the network training, for speeding up the network training process and improving the accuracy and interpretability of the model prediction. PINN successfully integrates the physical information with neural networks. We consider the following forward problem of a PDE with the Dirichlet boundary condition: where u denotes the solution of the equation, D x is the diferential operator respect to x, λ is the parameter in the governing equation, which is a known constant in the forward problem, Ω ∈ R and δΩ denotes the boundary, u 0 (x) is the initial condition at t � t 0 , and g(t, x) is the Dirichlet boundary condition. A typical PINN framework for solving the forward problem is shown in Figure 1. Te input training points (x, t) consist of three parts as follows: initial sampling point (x ic , 0), boundary sampling points (x bc , t bc ), and collocation points (x f , t f ) in the equation domain. Te predicted value is calculated using a fully connected feedforward DNN corresponding to the input point. Te symbol θ is the parameter set of the DNN, including weights W, bias b, and the activation function σ. Te automatic diferentiation of the DNN is utilized to calculate the partical derivatives of u NN (t, x; θ) with respect to x i and t i . Te loss function is evaluated using the contributions from the initial boundary conditions and the residual from the governing equation given by the physics-informed part. Ten, one seeks the optimal values of W and b to minimize the loss function below a certain specifed tolerance δ or until a prescribed maximum number n of iterations.
It is implemented by imposing three types of losses. One is the loss for governing equation learning l r controlled by the collocation points N r , the second is the loss for initial condition learning l ic calculated on the initial points N ic , and the last is the loss for boundary condition learning l bc calculated on the boundary points N bc . To combat overftting, the loss l r acts as a regularization mechanism that penalizes solutions that do not satisfy the governing equation. Consequently, PINN classifes the training points into two categories. One kind is the points in the space-time domain and the other is the initial boundary points. Unlike traditional numerical methods, to ft the initial and boundary conditions, PINN uses value constraints to train the neural network, which implies that there are errors in the learning of the initial and boundary conditions. Te loss function defned by the L 2 norm is then as follows: where where D (x) represents the learned spatial diferential operator, u NN (t, x; θ) is the learned solution, N r , N ic , and N bc represent, respectively, the internal confguration of the sampling point data , and r(t, x; θ) is the residual of the PDE. Te locations of the collocation points are generated by a spaceflling Latin hypercube sampling (LHS) strategy [48] and the initial boundary points are selected randomly.
PINN can also be applied to the inverse problem to discover the unknown parameters λ in equation (4). Inverse problems no longer require initial boundary values but rather the observed data in the space-time domain. Tey are solved on the same footing as forward problems, in which cases, the loss function consists of two parts. One is the loss for governing equation learning and the other is for observed data learning. Te loss function l is then defned by the following: in which where l m and l r are the mean square errors of the residuals for the measured data and the governing equation, respectively, N m is the measured data size, and u NN (t i m , x i m ; θ; λ) and u m (t i m , x i m ) are the predicted and measured values at the measuring points (t i m , x i m ). In the multicomponent CDR system, the reactor model described by a system of PDE is embedded into the loss of PINN for training. Te neural network optimizes not only the loss function of the network itself during the training iterations but also the residuals of each iteration of the governing equations, so that the results obtained from the ftting better satisfy the reaction laws. In the forward problem, no manually labeled reactant concentration data are required. Te PINN solves the problem by only providing the governing equations and the initial boundary conditions. In the inverse problem, the governing equation and information about the measurement points are encoded into the loss function for training. Te labeled data about the unknown parameter λ are not needed either. Te optimal model parameter set Θ * is obtained by minimizing the loss function. Te Adam optimization algorithm [23] is used in this paper to avoid the training process falling into the local optimum.

Numerical Results
In this section, numerical experiments on the nonlinear CDR systems presented in Section 2, are conducted to illustrate the capability and efciency of the developed PINN presented in Section 3. Te forward problems of the gassolid adsorption and the autocatalytic reacting fow are addressed frst in Sections 4.1 and 4.2, respectively, and the inverse problem of the autocatalytic reacting fow is then pursued in Section 4.3.
Te reference solutions are given by the fnite volume method [12], including the weighted essentially nonoscillation (WENO) scheme [44] and the modifed total variation diminishing Lax-Friedrichs scheme with Superbee limiter (MTVDLF-Superbee) [49]. Te solution of WENO is used as the reference for the gas-solid adsorption problem because it can efectively suppress the unphysical oscillations at the steep fronts. Te reference solution for the autocatalytic reacting fow problem is given by the MTVDLF-Superbee scheme. It can eliminate the numerical dissipation and spurious oscillations and is considered as an optimal method for handling the CDR problems [49].
Due to the similar computational complexity of the two models, the network structure is set to be the same in all the tests. Referring to the cases in the references [23,43], the network structure in this study is as follows: seven hidden layers and 100 neurons in each layer. More hidden layers and more neurons have been tested, but no signifcant diferences were observed. In the experiments, the used optimizer is Adam with a typical rate of 0.001. Te used software programs are TensorFlow 1.8.0 and Python 3.6, and the experiments are conducted on a platform with NVIDIA TITAN V and Intel (R) Xeon (R) Silver 4210 CPU at 2.20 GHz.

Forward Gas-Solid Adsorption Problem.
In this test, we consider whether PINN can simulate the dynamic behavior of a multireactant system with the given initial and boundary conditions. Te following parameters are set: ϵ � 0.4, v � 0.1 m/s, k � 0.0129/s, and K � 0.85. Te column length (L) is equal to 1.5 m.
Te initial conditions are as follows: Te Dirichlet boundary condition at X � 0 is as follows: Te discontinuous profle given by the initial condition moves continuously along the axial direction. Te reference solution, based on the result of the WENO-Roe-5 scheme on 300 fxed-grids [44], at t � 10s is as follows:

International Journal of Intelligent Systems
In this problem, the loss consists of three parts, namely, the initial concentration of the gas, the gas concentration at the boundary, and the governing equation for the gas-solid coupling reaction. Without manually labeled data, the concentrations of gas and solid reactants are learned by constraining the loss with the physical information given by the gas-solid adsorption model. During the training process, 100 initial points and 100 boundary points are randomly selected, and 2000 collocation points are generated by LHS in the space-time domain.
Te PINN solution is compared with the reference solution in Figure 2, and the results show that PINN accurately captures the dynamic behavior of the gas-solid adsorption column. Te running time is 0.36 h. To verify the stability of the algorithm, a set of 10 test errors has been obtained by 10 independent repetitions for this problem. Te error is given by the relative root mean square error (RMSE) between the PINN and the reference solution. Ten, the mean and standard deviation of the error are calculated, which is 2.16e − 02 ± 1.12e − 02 for C G and 3.41e − 02 ± 1.52e − 02 for C S , respectively. As shown in Figure 3, by increasing the surface gas velocity from 0.1 m/s to 0.2 m/s, the reacting reactants are accelerated to the boundary. To study the effects of void fraction ϵ on the concentration distribution, fve cases with ϵ � 0.8, 0.4, 0.2, 0.1, and 0.05 are studied, and the results are shown in Figure 4. It is seen that, as the void fraction decreases, the concentration no longer uniformly decreases along the range, but gradually shows a nonlinear trend, and the closer the concentration is to the inlet boundary, the faster is the decrease rate.
Te initial conditions are as follows: Te Dirichlet boundary conditions at X � 0 are as follows: Te Neumann boundary conditions at X � 1 are as follows: In this problem, the loss also consists of three parts, including the concentrations of the three reactants at the initial moment, the Dirichlet boundary condition at X � 0, and the Neumann boundary condition at X � 1, and the governing equation for the autocatalytic reacting fow. Te PINN results as function of X at T � 0.1 and 0.5 and function of T at X � 0.1 and 0.5 are shown in Figure 5, compared with the reference solution given by MTVDLF-Superbee [49]. Te good agreement is clearly observed. For this test, the running time is 0.46 h.
Since the sin activation function can improve the learning ability of the network for high-frequency signals [50,51], it is used in the above tests. Te tanh is also applied as the activation function and the results are shown in Figure 6, together with the PINN solution with activate function sin. Numerical oscillations at the boundary are observed. Terefore, the sin activation function is also better to ft high-frequency solutions of the CDR system studied here.
It is known that the Péclet number (Pe) defned by Pe i � v/D i has an important efect on the solution of a numerical method. For convection-dominated transport (i.e., Pe ≫ 1), the numerical solution can develop spurious oscillations (over or undershoot) or numerical dispersion [52,53]. Its efect on the solution of PINN is also studied here. Figure 7 shows the reactant concentrations over the entire space-time domain for the cases of Pe � 100, 1000 and 10000, and Figure 8 exhibits the simulated concentration distribution at the reactor center (X � 0.5) of the three reactants. Te sharpness of the moving fronts increases with the increasing Pe number. Even for the high Pe case (Pe � 10000), the PINN method still converges and captures the steep gradient with no oscillations. Terefore, the PINN method has good accuracy and efectiveness in solving the autocatalytic reacting fow problems.

Inverse Autocatalytic Reacting Flow Problem.
In this section, the application of PINN for the inverse autocatalytic reacting fow problems is investigated. In the forward problem, with the given initial and boundary conditions, accurate solutions of the three reactants have been obtained. In the inverse problem, the solutions of the autocatalytic reacting fow model are known at a given number of measuring points across the problem domain, while the model parameters λ � [α, β, c, v, D i ] T are unknown. Te measured data are encoded as constraints into the loss function of the neural network to identify the unknown parameters and to estimate the solution in the entire spacetime domain.
As no measured data is available, to illustrate the effectiveness of PINN, the dataset was generated by MTVDLF-Superbee [49] with α � 0.065, β � 2.0, c � 0.025, v � 1.0, and D 1 � D 2 � D 3 � 0.05. Diferent from the original PINN, with randomly obtained points in the space-time domain, an arrangement closer to the real situation is considered. Te monitoring distance interval is set to be 0.025 m, and the monitoring time interval is 0.025 s, resulting in a total number of 1600 data pairs 〈x, t, u〉.  [27].
In addition to providing predictions in the space-time domain, the solution to the inverse problem involves identifying the unknown parameters λ � [α, β, c, v, D i ] T . Table 1 presents all the model parameters learned by PINN from the supposed observations. Except β, all parameters have been correctly identifed with relative error less than  International Journal of Intelligent Systems 5%. Te parameters associated with β are α and D 3 , resulting in a relative high error of 1% for α and 4% for D 3 . Other parameters are identifed with relative errors less than 0.5%. Tis test demonstrates the excellent capability of PINN for identifying the parameters in the CDR system. Note that the error is given by the RMSE between the predicted and the reference value defned by the following: Table 2 presents the learned kinetics of the reaction and mutation efciency, supposing the fow velocity and difusion coefcients are known. When only the reaction-related model parameters are predicted, the relative errors for both α and c are less than 1%, but it remains large for β. Tat is, when PINN is used to identify fewer model parameters, the prediction accuracy will be improved as expected.
From the two tables, it is seen that all the reaction parameters have been correctly learned except the mutation efciency β. Te large deviance in the recognition of β is attributed to the fact that it is insensitive to the governing equation. To verify this, the forward problem with three diferent β was solved using PINN and the results are shown in Figure 10. Te concentrations of substrate A and autocatalyst B do not change when taking diferent values of β (see Figures 10(a) and 10(b)). However, due to the small magnitude of the mutant C, a slight diference in the concentration of reactant C at T � 1 is found (Figure 10(c)).          In reality, there is often noise in the measured data, which leads to various difculties for the learning methods and highly afects the recognition accuracy of these parameters. Te capacity of the PINN to solve the inverse problem with noisy data was investigated by adding Gaussian noise to the dataset. Table 3 shows the error in the PINN solution of the inverse problem at diferent noise levels (noise � 0%, 1%, 5%, 10%) and diferent sizes of the measured point (N � 1000, 2000, 3000), supposing the fow velocity and difusion coefcients are known. Te results indicate that the quality of the prediction over the problem domain decreases with increasing noise levels from error ≈ 10 − 3 for noise � 0% to error ≈ 10 − 2 for noise � 10%. However, the identifcations of the model parameters do not show signifcant variations due to increasing noise levels.
Te proposed method appears to be robust to noise levels in the data, and a reasonable recognition accuracy is maintained even for noise corruptions up to 10%. At the same time, the error of parameter learning is hardly afected by the size of the training data. Tis means that for CDR systems, PINN can obtain accurate results with sparse training data.

Conclusion
In this paper, a PINN framework for solving the forward and inverse problems of nonlinear CDR systems is presented. In PINN, the CDR systems expressed as PDEs are incorporated into the neural network. Due to being devoid of grids or nodes, PINN is a mesh-free method that can predict the solution at any point in the equation domain without interpolation.
For the forward gas-solid adsorption problem, the reaction process is simulated for diferent convection velocities and void fractions. Te PINN results agree well with the reference solutions, and the moving steep fronts are correctly captured without numerical dissipation and spurious oscillations.
For the forward autocatalytic reacting fow problem, the PINN method accurately predicts the dynamic profles of the system. Compared with the standard activation function of tanh, the sin activation function can more efectively eliminate the unphysical oscillation generated at the boundary. For high Péclet numbers, PINN still captures the sharp profles without any unphysical oscillation or numerical dissipation.
For the inverse autocatalytic reacting fow problem, with limited data, PINN successfully identifed the unknown parameters in the CDR systems, even the measured data are heavily polluted by noise. Tis demonstrates a strong ability of PINN to learn missing chemical information and to better observe and explain the laws in the reaction chemistry.

Data Availability
Te data that support the fndings of this study are available from the corresponding author upon reasonable request.

Conflicts of Interest
Te authors declare that they have no conficts of interest.