The Relationship between the Stochastic Maximum Principle and the Dynamic Programming in Singular Control of Jump Diffusions

The main objective of this paper is to explore the relationship between the stochastic maximum principle (SMP in short) and dynamic programming principle (DPP in short), for singular control problems of jump diffusions. First, we establish necessary as well as sufficient conditions for optimality by using the stochastic calculus of jump diffusions and some properties of singular controls. Then, we give, under smoothness conditions, a useful verification theorem and we show that the solution of the adjoint equation coincides with the spatial gradient of the value function, evaluated along the optimal trajectory of the state equation. Finally, using these theoretical results, we solve explicitly an example, on optimal harvesting strategy, for a geometric Brownian motion with jumps.


Introduction
In this paper, we consider a mixed classical-singular control problem, in which the state evolves according to a stochastic differential equation, driven by a Poisson random measure and an independent multidimensional Brownian motion, of the following form:   =  (,   ,   )  +  (,   ,   )   + ∫   (,  − ,   , ) Ñ (, ) +     ,  0 = , (1) where , , , and  are given deterministic functions and  is the initial state.The control variable is a suitable process (, ), where  : [0,] × Ω →  1 ⊂ R  is the usual classical absolutely continuous control and  : [0, ] × Ω →  2 = ([0, ∞))  is the singular control, which is an increasing process, continuous on the right with limits on the left, with  0− = 0.The performance functional has the form (2) The objective of the controller is to choose a couple ( ⋆ ,  ⋆ ) of adapted processes, in order to maximize the performance functional.
In the first part of our present work, we investigate the question of necessary as well as sufficient optimality conditions, in the form of a Pontryagin stochastic maximum principle.In the second part, we give under regularity assumptions, a useful verification theorem.Then, we show that the adjoint process coincides with the spatial gradient of the value function, evaluated along the optimal trajectory of the state equation.Finally, using these theoretical results, we solve explicitly an example, on optimal harvesting strategy for a geometric Brownian motion, with jumps.Note that our results improve those in [1,2] to the jump diffusion setting.Moreover we generalize results in [3,4], by allowing 2 International Journal of Stochastic Analysis both classical and singular controls, at least in the complete information setting.Note that in our control problem, there are two types of jumps for the state process, the inaccessible ones which come from the Poisson martingale part and the predictable ones which come from the singular control part.The inclusion of these jump terms introduces a major difference with respect to the case without singular control.
Stochastic control problems of singular type have received considerable attention, due to their wide applicability in a number of different areas; see [4][5][6][7][8].In most cases, the optimal singular control problem was studied through dynamic programming principle; see [9], where it was shown in particular that the value function is continuous and is the unique viscosity solution of the HJB variational inequality.
The one-dimensional problems of the singular type, without the classical control, have been studied by many authors.It was shown that the value function satisfies a variational inequality, which gives rise to a free boundary problem, and the optimal state process is a diffusion reflected at the free boundary.Bather and Chernoff [10] were the first to formulate such a problem.Beneš et al. [11] explicitly solved a one-dimensional example by observing that the value function in their example is twice continuously differentiable.This regularity property is called the principle of smooth fit.The optimal control can be constructed by using the reflected Brownian motion; see Lions and Sznitman [12] for more details.Applications to irreversible investment, industry equilibrium, and portfolio optimization under transaction costs can be found in [13].A problem of optimal harvesting from a population in a stochastic crowded environment is proposed in [14] to represent the size of the population at time  as the solution of the stochastic logistic differential equation.The two-dimensional problem that arises in portfolio selection models, under proportional transaction costs, is of singular type and has been considered by Davis and Norman [15].The case of diffusions with jumps is studied by Øksendal and Sulem [8].For further contributions on singular control problems and their relationship with optimal stopping problems, the reader is referred to [4,5,7,16,17].
The stochastic maximum principle is another powerful tool for solving stochastic control problems.The first result that covers singular control problems was obtained by Cadenillas and Haussmann [18], in which they consider linear dynamics, convex cost criterion, and convex state constraints.A first-order weak stochastic maximum principle was developed via convex perturbations method for both absolutely continuous and singular components by Bahlali and Chala [1].The second-order stochastic maximum principle for nonlinear SDEs with a controlled diffusion matrix was obtained by Bahlali and Mezerdi [19], extending the Peng maximum principle [20] to singular control problems.A similar approach has been used by Bahlali et al. in [21], to study the stochastic maximum principle in relaxed-singular optimal control in the case of uncontrolled diffusion.Bahlali et al. in [22] discuss the stochastic maximum principle in singular optimal control in the case where the coefficients are Lipschitz continuous in , provided that the classical derivatives are replaced by the generalized ones.See also the recent paper by Øksendal and Sulem [4], where Malliavin calculus techniques have been used to define the adjoint process.
Stochastic control problems in which the system is governed by a stochastic differential equation with jumps, without the singular part, have been also studied, both by the dynamic programming approach and by the Pontryagin maximum principle.The HJB equation associated with this problems is a nonlinear second-order parabolic integrodifferential equation.Pham [23] studied a mixed optimal stopping and stochastic control of jump diffusion processes by using the viscosity solutions approach.Some verification theorems of various types of problems for systems governed by this kind of SDEs are discussed by Øksendal and Sulem [8].Some results that cover the stochastic maximum principle for controlled jump diffusion processes are discussed in [3,24,25].In [3] the sufficient maximum principle and the link with the dynamic programming principle are given by assuming the smoothness of the value function.Let us mention that in [24] the verification theorem is established in the framework of viscosity solutions and the relationship between the adjoint processes and some generalized gradients of the value function are obtained.Note that Shi and Wu [24] extend the results of [26] to jump diffusions.See also [27] for systematic study of the continuous case.The second-order stochastic maximum principle for optimal controls of nonlinear dynamics, with jumps and convex state constraints, was developed via spike variation method, by Tang and Li [25].These conditions are described in terms of two adjoint processes, which are linear backward SDEs.Such equations have important applications in hedging problems [28].Existence and uniqueness for solutions to BSDEs with jumps and nonlinear coefficients have been treated by Tang and Li [25] and Barles et al. [29].The link with integral-partial differential equations is studied in [29].
The plan of the paper is as follows.In Section 2, we give some preliminary results and notations.The purpose of Section 3 is to derive necessary as well as sufficient optimality conditions.In Section 4, we give, under-regularity assumptions, a verification theorem for the value function.Then, we prove that the adjoint process is equal to the derivative of the value function evaluated at the optimal trajectory, extending in particular [2,3].An example has been solved explicitly, by using the theoretical results.

Assumptions and Problem Formulation
The purpose of this section is to introduce some notations, which will be needed in the subsequent sections.In all what follows, we are given a probability space (Ω, F, (F  ) ≤ , P), such that F 0 contains the P-null sets, F  = F for an arbitrarily fixed time horizon , and (F  ) ≤ satisfies the usual conditions.We assume that (F  ) ≤ is generated by a -dimensional standard Brownian motion  and an independent jump measure  of a Lévy process , on [0, ] × , where  ⊂ R  \ {0} for some  ≥ 1.We denote by (F   ) ≤ (resp., (F   ) ≤ ) the P-augmentation of the natural filtration of  (resp., ).We assume that the compensator of  has the form (, ) = ](), for some -finite Lévy measure ] on , endowed with its Borel -field B().We suppose that ∫  1 ∧ || 2 ]() < ∞ and set Ñ(, ) = (, ) − ](), for the compensated jump martingale random measure of .
Obviously, we have where N denotes the totality of ]-null sets and  1 ∨  2 denotes the -field generated by  1 ∪  2 .
Notation.Any element  ∈ R  will be identified with a column vector with  components, and its norm is The scalar product of any two vectors  and  on R  is denoted by  or ∑  =1     .For a function ℎ, we denote by ℎ  (resp., ℎ  ) the gradient or Jacobian (resp., the Hessian) of ℎ with respect to the variable .
Given  < , let us introduce the following spaces.
Assume that, for (, ) ∈ U,  ∈ [0, ], the state   of our system is given by where  ∈ R  is given, representing the initial state. Let be measurable functions.
Notice that the jump of a singular control  ∈ U 2 at any jumping time  is defined by Δ  =   −  − , and we let be the continuous part of .
We distinguish between the jumps of   caused by the jump of (, ), defined by The expression (12) defines the jump in the value of (  ) caused by the jump of  at .We emphasize that the possible jumps in   coming from the Poisson measure are not included in Δ  (  ).
Let ( ⋆ ,  ⋆ ) be an optimal control and let  ⋆ be the corresponding optimal trajectory.Then, we consider a triple (, , (⋅)) of square integrable adapted processes associated with 3.1.Necessary Conditions of Optimality.The purpose of this section is to derive optimality necessary conditions, satisfied by an optimal control, assuming that the solution exists.The proof is based on convex perturbations for both absolutely continuous and singular components of the optimal control and on some estimates of the state processes.Note that our results generalize [1,2,21] for systems with jumps.
Theorem 2 (necessary conditions of optimality).Let ( ⋆ ,  ⋆ ) be an optimal control maximizing the functional  over U, and let  ⋆ be the corresponding optimal trajectory.Then there exists an adapted process , which is the unique solution of the BSDE (18), such that the following conditions hold.
In order to prove Theorem 2, we present some auxiliary results..Let (V, ) ∈ U be such that ( ⋆ + V,  ⋆ + ) ∈ U.The convexity condition of the control domain ensures that for  ∈ (0, 1) the control ( ⋆ +V,  ⋆ +) is also in U. We denote by   the solution of the SDE (8) corresponding to the control ( ⋆ + V,  ⋆ + ).Then by standard arguments from stochastic calculus, it is easy to check the following estimate.

Lemma 3. Under assumptions (H 1 )-(H 5 ), one has
Proof.From assumptions (H 1 )-(H 5 ), we get by using the Burkholder-Davis-Gundy inequality From Definition 1 and Gronwall's lemma, the result follows immediately by letting  go to zero.
We define the process   =   ⋆ ,V,  by From (H 2 ) and Definition 1, one can find a unique solution  which solves the variational equation ( 26), and the following estimate holds.

Lemma 4. Under assumptions (H 1 )-(H 5 ), it holds that
Proof.Let We denote  ,  =  ⋆  + (Γ   +   ) and  ,  =  ⋆  + V  , for notational convenience.Then we have immediately that Γ  0 = 0 and Γ   satisfies the following SDE: Since the derivatives of the coefficients are bounded, and from Definition 1, it is easy to verify by Gronwall's inequality that Γ  ∈ S where    is given by Since   ,   , and   are bounded, then where  is a generic constant depending on the constants , ](), and .We conclude from Lemma 3 and the dominated convergence theorem, that lim  → 0    = 0. Hence (27) follows from Gronwall's lemma and by letting  go to 0. This completes the proof.

Variational Inequality.
Let Φ be the solution of the linear matrix equation, for 0 ≤  <  ≤ where   is the  ×  identity matrix.This equation is linear, with bounded coefficients, then it admits a unique strong solution.Moreover, the condition (H 4 ) ensures that the tangent process Φ is invertible, with an inverse Ψ satisfying suitable integrability conditions.From Itô's formula, we can easily check that (Φ , Ψ , ) = 0, and Φ , Ψ , =   , where Ψ is the solution of the following equation so Ψ = Φ −1 .If  = 0 we simply write Φ 0, = Φ  and Ψ 0, = Ψ  .By the integration by parts formula ([8, Lemma 3.6]), we can see that the solution of ( 26) is given by   = Φ    , where   is the solution of the stochastic differential equation Let us introduce the following convex perturbation of the optimal control ( ⋆ ,  ⋆ ) defined by for some (V, ) ∈ U and  ∈ (0, 1).Since ( ⋆ ,  ⋆ ) is an optimal control, then  −1 ((  ,   ) − ( ⋆ ,  ⋆ )) ≤ 0. Thus a necessary condition for optimality is that lim The rest of this subsection is devoted to the computation of the above limit.We will see that the expression (37) leads to a precise description of the optimal control ( ⋆ ,  ⋆ ) in terms of the adjoint process.First, it is easy to prove the following lemma.

Lemma 5. Under assumptions (H 1 )-(H 5 ), one has
Proof.We use the same notations as in the proof of Lemma 4. First, we have where By using Lemma 4, and since the derivatives   ,   , and   are bounded, we have lim  → 0    = 0.Then, the result follows by letting  go to 0 in the above equality.

Adjoint Equation and Maximum
Principle.Since (37) is true for all (V, ) ∈ U and  ≤ 0, we can easily deduce the following result.
Theorem 7. Let ( ⋆ ,  ⋆ ) be the optimal control of the problem (14) and denote by  ⋆ the corresponding optimal trajectory, then the following inequality holds: where the Hamiltonian  is defined by (17), and the adjoint variable (,   , (⋅)) for  = 1, . . ., , is given by (44).Now, we are ready to give the proof of Theorem 2.
Proof of Theorem 2. (i) Let us assume that ( ⋆ ,  ⋆ ) is an optimal control for the problem ( 14), so that inequality (48) is valid for every (V, ).If we choose  =  ⋆ in inequality (48), we see that for every measurable, F  -adapted process If  ⊗ P( V ) > 0, where  denotes the Lebesgue measure, then which contradicts (49), unless  ⊗ P( V ) = 0. Hence the conclusion follows.
(ii) If instead we choose V =  ⋆ in inequality (48), we obtain that for every measurable, F  -adapted process  : [0, ] × Ω →  2 , the following inequality holds: By comparing with (53) we get Next, let  be defined by Then, the relation (53) can be written as which implies that By the fact that    +    ( − + Δ    ) < 0, and Δ   ≥ 0, we get Thus ( 23) holds.The proof is complete.Now, by applying Itô's formula to  ⋆  Ψ  , it is easy to check that the processes defined by relation (44) satisfy BSDE (18) called the adjoint equation.

Sufficient Conditions of Optimality.
It is well known that in the classical cases (without the singular part of the control), the sufficient condition of optimality is of significant importance in the stochastic maximum principle, in the sense that it allows to compute optimal controls.This result states that, under some concavity conditions, maximizing the Hamiltonian leads to an optimal control.
In this section, we focus on proving the sufficient maximum principle for mixed classical-singular stochastic control problems, where the state of the system is governed by a stochastic differential equation with jumps, allowing both classical control and singular control.
Proof.For convenience, we will use the following notations throughout the proof: (65) Let (, ) be an arbitrary admissible pair, and consider the difference We first note that, by concavity of , we conclude that which implies that By the fact that (,   , (⋅)) ∈ S 2 × M 2 × L 2 ] for  = 1, . . ., , we deduce that the stochastic integrals with respect to the local martingales have zero expectation.Due to the concavity of the Hamiltonian , the following holds The definition of the Hamiltonian  and (64) leads to ( ⋆ ,  ⋆ )−(, ) ≥ 0, which means that ( ⋆ ,  ⋆ ) is an optimal control for the problem (14).
The expression (64) is a sufficient condition of optimality in integral form.We want to rewrite this inequality in a suitable form for applications.This is the objective of the following theorem which could be seen as a natural extension of [2, Theorem 2.2] to the jump setting and [3, Theorem 2.1] to mixed regular-singular control problems.
Proof.Using (71) and ( 72) yields The same computations applied to (73) and ( 74) imply Hence, from Definition 1, we have the following inequality: The desired result follows from Theorem 8.

Relation to Dynamic Programming
In this section, we come back to the control problem studied in the previous section.We recall a verification theorem, which is useful to compute optimal controls.Then we show that the adjoint process defined in Section 3, as the unique solution to the BSDE (18), can be expressed as the gradient of the value function, which solves the HJB variational inequality.

A Verification Theorem. Let 𝑥 𝑡,𝑥
be the solution of the controlled SDE (8), for  ≥ , with initial value   = .To put the problem in a Markovian framework, so that we can apply dynamic programming, we define the performance criterion Since our objective is to maximize this functional, the value function of the singular control problem becomes  (, ) = sup (,)∈U  (,) (, ) . (79) If we do not apply any singular control, then the infinitesimal generator A  , associated with (8), acting on functions , coincides on  2  (R  ; R) with the parabolic integrodifferential operator A  given by where 1 and   2 , for  = 1, . . ., , are given by We start with the definition of classical solutions of the variational inequality (81).
(, ) + A   (, ) +  (, , ) ≤ 0, The following verification theorem is very useful to compute explicitly the value function and the optimal control, at least in the case where the value function is sufficiently smooth.
In the following, we present an example on optimal harvesting from a geometric Brownian motion with jumps see, for example, [5,8].
Example 12. Consider a population having a size  = {  :  ≥ 0} which evolves according to the geometric Lévy process; that is Here   is the total number of individuals harvested up to time .If we define the price per unit harvested at time  by () =  − and the utility rate obtained when the size of the population at  is   by  −    .Then the objective is to maximize the expected total time-discounted value of the harvested individuals starting with a population of size ; that is, where  := inf{ ≥ 0 :   = 0} is the time of complete depletion,  ∈ (0, for 0 <  < .We try a solution Φ of the form hence where Ψ is the fundamental solution of the ordinary integrodifferential equation We notice that Ψ() =   +  , for some arbitrary constant ; we get where Note that ℎ 1 (1) =  −  < 0 and lim  → ∞ ℎ 1 () = ∞; then there exists  > 1 such that ℎ 1 () = 0.The constant  is given by Outside  we require that Ψ() =  + , where  is a constant to be determined.This suggests that the value must be of the form Assuming smooth fit principle at point , then the reflection threshold is where Since  < 1 and  > 1, we deduce that  > 0.
To construct the optimal control  ⋆ , we consider the stochastic differential equation ⋆  ≤ ,  ≥ 0, (107) and if this is the case, then Arguing as in [7], we can adapt Theorem 15 in [16] to obtain an identification of the optimal harvesting strategy as a local time of a reflected jump diffusion process.Then, the system (106)-( 109 where  ⋆ is defined as in Theorem 11, and the sum is taken over all jumping times  of  ⋆ .Note that where Δ ⋆  =  ⋆  −  ⋆ − is a pure jump process.Then, we can rewrite (114) as follows: (, ) =  [∫  0  (,   ,   )  + ∫  0  ()   +  (  )] .