Optimal Control of a Nonlinear Time-Delay System in Batch Fermentation Process

The main control goal in batch process is to get a high yield of products. In this paper, to maximize the yield of 1,3-propanediol (1,3-PD) in bioconversion of glycerol to 1,3-PD, we consider an optimal control problem involving a nonlinear time-delay system. The control variables in this problem include the initial concentrations of biomass and glycerol and the terminal time of the batch process. By a time-scaling transformation, we transcribe the optimal control problem into a new one with fixed terminal time, which yields a new nonlinear system with variable time-delay. The gradients of the cost and constraint functionals with respect to the control variables are derived using the costate method. Then, a gradient-based optimization method is developed to solve the optimal control problem. Numerical results show that the yield of 1,3-PD at the terminal time is increased considerably compared with the experimental data.


Introduction
Batch processing represents the natural way to scale-up processes from the laboratory to the production environment [1].During the batch operation, no substrate is added to the initial charge and no product is removed until the end of the process [2].Optimal control of batch processes has received attention recently because it is a choice for reducing production costs, improving product quality, and meeting safety requirements [3][4][5].
1,3-Propanediol (1,3-PD) is one of the important products used in the chemical industry.Using glycerol for producing 1,3-PD is effective from both the economical and ecological point of view.The operations of glycerol bioconversion to 1,3-PD consist of batch, continuous, and fed-batch cultures.Compared with continuous and fed-batch cultures, glycerol fermentation in batch process can obtain the highest production concentration and molar yield 1,3-PD to glycerol [6].For this process, many studies have been carried out including the quantitative description of the cell growth kinetics [7,8], enzyme-catalytic kinetics [9], the parameter identification problem [10][11][12][13], and the pathway optimization problem [14].In particular, considering the existence of time-delays in the fermentation process [15,16], a nonlinear time-delay system was recently proposed in [13].Numerical simulations showed that the nonlinear time-delay system can describe the batch process better than previous mathematical models.However, in batch process of glycerol bioconversion to 1,3-PD, the aim is to obtain as much 1,3-PD as possible at the terminal time.Mathematically, this is an optimal control problem.Although the achieved results are interesting, such optimal control problem is ignored in the above researches.
In this paper, taking the initial concentrations of biomass and glycerol and the terminal time of the fermentation process as control variables and the yield of 1,3-PD as the performance index, we propose an optimal control model involving nonlinear time-delay system in [13] and subject to continuous state constraints.In fact, this problem is a free time delayed optimal control problem.By the way, there has been a mounting interest in optimal control of timedelay systems.A maximum principle for optimal control problem with a constant delay was provided in [17].It should, however, be noted that many time-delay systems including the nonlinear time-delay system in this paper are highly nonlinear.Therefore, it is often impossible to obtain analytical solutions of the delayed optimal control problem and one has to resort to numerical solution methods.As a result, some successful algorithms, such as iterative dynamic programming method [18], control parameterization method [19], and measure theoretical approach [20], have been developed.However, terminal time in above optimal control problems is fixed.The presence of free terminal time makes the delayed optimal control problem much more complicated.Thus, to solve the optimal control problem in this work, these existing computational methods cannot be used directly and new computational techniques should be explored.
In this paper, by a time-scaling transformation, we equivalently transcribe the free time delayed optimal control problem into one with fixed terminal time.It is significant to mention that this transformation yields a more complex dynamic system in which the time-delays are variable.By the costate method, we derived the gradients of the cost functional and constraints with respect to the control variables.Then, a gradient-based optimization technique is developed.Finally, numerical results show that the yield of 1,3-PD at the terminal time is increased considerably compared with the experiment data.
The terminal time in system (1) is a control variable.Define where  min and  max are the lower and upper bounds for the terminal time, respectively.Any  ∈ T is called an admissible terminal time.Moreover, since no product comes into being at the initial point,  ℓ = 0, ℓ = 3, 4, 5. Hence, let  = ( 1 ,  2 ) ⊤ and define where   and   are real numbers such that   <   .Any  ∈ U is called an admissible initial vector of biomass and glycerol.
For the nonlinear time-delay system (1), there exists a unique continuous solution (⋅ | ) corresponding to each  ∈ U on [0, ∞) [21].Furthermore, there exist critical concentrations of biomass, glycerol, and products, outside which cells cease to grow.Hence, it is biologically meaningful to restrict the concentrations of biomass, glycerol, and products within a set  defined as

Optimal Control Problems
In the batch process, it is desired that the yield of 1,3-PD should be maximized at the end of the process.This is achieved by manipulating some control variables  and .Thus, the optimal control problem in batch process can be formulated as Note that (OCP) is of nonstandard type because the terminal time and the initial values of biomass and glycerol are variables to be determined.Thus, the (OCP) is actually a free time optimal control problem.It is difficult to solve the (OCP) using existing numerical techniques [18][19][20].The main difficulty is the implicit dependence of the system state on the terminal time.We now employ a time-scaling transformation from [0, ] to [0, 1] as follows: where Mathematical Problems in Engineering 3 The initial conditions become x (0) = , x () =  (, ) ,  ≤ 0. (10) Thus, system (1) is transformed into the following form: x (0) = , x () =  (, ) ,  ≤ 0.
Let x(⋅ | , ) denote the solution of system (11) corresponding to each (, ) ∈ U × T on [0, 1].Then, (OCP) can be transcribed into the following equivalently optimal control problem with fixed terminal time: Note that the time-delay in system (11) depends on the control variable .Thus, when applied to time-delay systems, the time-scaling transformation (8) yields a more complex dynamic system in which the time-delay is variable.

Computational Approaches
(EOCP) is in essence an optimization problem with continuous state inequality constraint (13), which has an infinite number of constraints and can be viewed as a semi-infinite programming problem.An efficient algorithm transforming this type of problems to standard mathematical programming problems was discussed in [22].We will now briefly discuss the application of this algorithm to (EOCP). Let The continuous state constraint (13) becomes where (, ) := ∑ 10 =1 ∫ 1 0 min{0,   ( x( | , ))}.However, the equality constraint ( 15) is nondifferentiable at the points when   = 0. We replace (15) with the following inequality constraint: (16) where  > 0,  > 0, and Thus, (EOCP) is approximated by a sequence of standard mathematical programming problems {(EOCP , )} defined by replacing constraint (15) with (16).Moreover, each of {(EOCP , )} can be solved by a gradient-based optimization method (e.g., sequential quadratic programming (SQP) [23]).However, this optimization method requires the gradients of the cost functional and constraints.Now, we will derive these required gradients using the so-called costate method, which is a commonly used technique in the optimal control domain [22,24,25].Define The gradients of the cost functional (⋅, ⋅) defined in (12) with respect to the control variables are given in the following theorem.
Let  : [0, ∞] →  5 be an arbitrary function that is continuous and differentiable almost everywhere.Then, we may express the cost functional  as follows: Applying integration by parts to the last integral term gives Differentiating ( 24) with respect to  yields Performing a change of variable in the last term on the righthand side of (25) gives Substituting ( 26) into (25) yields Choosing (⋅) = (⋅) and substituting ( 21)-( 22) into the above equation, we obtain the conclusion (19).The gradient formula (20) can be derived similarly.The proof is complete.
The gradients of the constraint G, (⋅, ⋅) defined in ( 16) with respect to the control variables are given in the next theorem.
Proof.The derivations of the gradients of the constraint G, (, ) with respect to  and  are similar.Thus, only the derivation of the gradient of the constraint G, (, ) with respect to  is given below.Let  : [0, ∞] →  5 be an arbitrary function that is continuous and differentiable almost everywhere.Then, we may express the constraint G, as follows: Choosing (⋅) = λ(⋅) and substituting (30) and (31) into the above equation, we obtain the conclusion (28).The gradient formula (29) can be derived similarly.Thus, the proof is complete.
On the basis of Theorems 1 and 2, we can develop the following algorithm to solve the (OCP).
Step 4. If ( * , ,  * , ) is feasible, then go to Step 5. Otherwise, set  := , where  is a given positive constant.If  < , where  is a prespecified positive constant, then go to Step 6.Otherwise, go to Step 2.
Step 5. Set  := , where  is a given positive constant.If  > , then go to Step 2. Otherwise, go to Step 6.

Numerical Results
Algorithm 1 was applied to seek the optimal control variables in (OCP) and all computations were implemented in Fortran  6.5.We obtained that  * = (0.201867, 418.376) ⊤ and  * = 7.2578 h.Here, nonlinear time-delay system (1) was solved using the six-order Runge-Kutta method with Lagrange interpolation [26].The time-delay ℎ = 0.26 h, the initial state  = (0.102, 418.26087, 0, 0, 0) ⊤ , the initial function () = , the terminal time  = 6.92 h, and the kinetic parameters and critical concentrations are listed in Table 1.In the optimization process, we assume that  1 ∈ [0.02, 0.202],  2 ∈ [400, 500], and  ∈ [5.0, 8.0].In addition, the smoothing and feasible parameters were initially selected as  = 0.1 and  = 0.01.The parameters  and  were chosen as 0.1 and 0.01 until the solution obtained was feasible for the (OCP).The process terminated when  = 1.0 × 10 −8 and  = 1.0 × 10 −7 .Under the obtained optimal control strategy, we obtained the optimal yield of 1,3-PD at the optimal terminal time is 78.1586 mmol h −1 , which is increased by 90.04% compared with experiment data [13].Furthermore, we plotted the optimal yield change of 1,3-PD with respect to the fermentation time in Figure 1.From Figure 1, we can see that the yield of 1,3-PD at the terminal time is actually increased.

Conclusion
In this paper, the optimal control problem in batch process was investigated.We presented the optimal control problem involving a nonlinear time-delay system and with free terminal time.We then transcribe the free time optimal control problem into a new one with fixed terminal time and variable time-delay.We developed a computational method based on the gradients of the cost and constraint functionals with respect to the control variables.Numerical results showed the effectiveness of the developed computational method.