A SINGULAR CONTROL PROBLEM WITH AN EXPECTED AND A PATHWISE ERGODIC PERFORMANCE CRITERION

We consider the problem of controlling a general one-dimensional Itô diffusion by means of a finite-variation process. The objective is to minimise a long-term average expected criterion as well as a long-term pathwise criterion that penalise deviations of the underlying state process from a given nominal point as well as the expenditure of control effort. We solve the resulting singular stochastic control problems under general assumptions by identifying an optimal strategy that is explicitly characterised.


Introduction
We consider a stochastic system, the state of which is modelled by the controlled, onedimensional It ô diffusion dX t = b X t dt + dξ t + σ X t dW t , X 0 = x ∈ R, (1.1) where W is a standard, one-dimensional Brownian motion, and the controlled process ξ is a càglàd finite-variation process.The objective of the optimisation problem is to minimise the long-term average expected performance criterion limsup as well as the long-term average pathwise performance criterion limsup over all admissible choices of ξ.Here, h is a given positive function that is strictly decreasing in ] − ∞, 0[ and strictly increasing in ]0, ∞[.Thus, these performance indices penalise deviations of the state process X from the nominal operating point 0. Given a finite-variation process ξ, we denote by ξ + and ξ − the unique processes that provide the minimal decomposition of ξ into the difference of two increasing processes, and where (ξ ± ) c are the continuous parts of ξ ± .The functions k + and k − represent the proportional costs associated with the use of the control process ξ to push the left-continuous state process X in the positive and negative directions, respectively.Singular stochastic control with an expected discounted criterion was introduced by Bather and Chernoff [1] who considered a simplified model of spaceship control.In their seminal paper, Beneš et al. [2] were the first to solve rigorously an example of a finite-fuel singular control problem.Since then, the area has attracted considerable interest in the literature.Karatzas [9], Harrison and Taksar [8], Shreve et al. [17], Chow et al. [3], Sun [19], Soner and Shreve [18], Ma [11], Zhu [21], and Fleming and Soner [7, Chapter VIII] provide an incomplete list, in chronological order, of further important contributions.
The first singular control problem with an ergodic expected criterion was solved by Karatzas [9], who considered the control of a standard Brownian motion.Menaldi and Robin [12] later established the existence of an optimal control to the problem that we consider when b and σ are Lipshitz continuous with σ bounded, k + ≡ k − ≡ 0, and under other technical conditions.Also, Weerasinghe [20] solved the version of the problem that arises when the drift is controllable in a bang-bang sense and the functions k + and k − are both equal to the same constant K.
At this point, we should note that, to the best of our knowledge, our analysis provides the first model of singular control with an ergodic pathwise criterion to be considered in the literature.Other stochastic control problems with an ergodic pathwise criterion have recently attracted significant interest.Notable contributions in this area include Rotar [15], Presman et al. [13], Dai Pra et al. [4], Dai Pra et al. [5], and a number of references therein.
We solve the problems that we consider and we derive an explicit characterisation of an optimal strategy by finding an appropriate solution to the associated Hamilton-Jacobi-Bellman (HJB) equation under general assumptions.When the cost functions k + and k − are both equal to a constant K, our assumptions regarding the rest of the problem data are similar to those imposed by Weerasinghe [20].However, we should note that, in this special case, the problem with the expected performance criterion that we solve is fundamentally different from the one solved by Weerasinghe [20], which is evidenced by the fact that the two problems are associated with different HJB equations.
With regard to the structure of the performance criteria that we consider, penalising the expenditure of control effort by means of integrals as in (1.4) was introduced by Zhu [21] and was later adopted by Davis and Zervos [6].An apparently more natural choice for penalising control effort expenditure would arise if we replaced these integrals by respectively.However, such a choice would lead to a less "pleasant" HJB equation that we cannot solve.Also, it is worth noting that the two types of integrals are identical when the functions k + and k − are both constant.At this point, we should note that our assumptions allow for the possibility that the uncontrolled diffusion associated with (1.1) explodes in finite time.In such a case, it turns out that an optimal control is a stabilising one.

The singular stochastic control problem
We consider a stochastic system, the state process X of which is driven by a Brownian motion W and a controlled process ξ.In particular, we consider the controlled, onedimensional SDE where b,σ : R → R are given functions, and W is a standard, one-dimensional Brownian motion.Here, the singular control process ξ is a càglàd finite-variation process, the time evolution of which is determined by the system's controller.Given such a process, we denote by ξ = ξ + − ξ − the unique decomposition of ξ into the difference of two increasing processes ξ + and ξ − such that the total variation process ξ of ξ is given by ξ = ξ + + ξ − .We adopt a weak formulation of the control problem that we study.
Definition 2.1.Given an initial condition x ∈ R a control of a stochastic system governed by dynamics as in (2.1) is an octuple C x = (Ω,Ᏺ,Ᏺ t ,P,W,ξ,X,τ), where (Ω,Ᏺ,Ᏺ t ,P) is a filtered probability space satisfying the usual conditions, W is a standard one-dimensional (Ᏺ t )-Brownian motion, ξ is a finite-variation (Ᏺ t )-adapted càglàd process, and X is an (Ᏺ t )-adapted càglàd process that satisfies (2.1) up to its possible explosion time τ.Define Ꮿ x to be the family of all such controls C x .
With each control C x ∈ Ꮿ x , we associate the long-term average expected performance criterion J E (C x ) defined by if P(τ = ∞) = 1, and by 4 Ergodic singular control as well as the long-term average pathwise criterion which is a random variable with values in [0, ∞].Here, h : R → R is a given function that models the running cost resulting from the system's operation, while k + , k − are given functions penalising the expenditure of control effort.The integrals with respect to ξ + and ξ − are defined by (1.4) in the introduction, respectively.The objective is to minimise the performance criteria defined by (2.2)-( 2.3) and (2.4) over all admissible controls C x ∈ Ꮿ x .We impose the following assumption on the problem data.
(a) The functions b, σ : R → R are continuous, and there exists a constant (b) The function h is continuous, strictly decreasing on ] − ∞, 0[ and strictly increasing on ]0, ∞[.Also, h(0) = 0 and there exists a constant C 2 > 0 such that (d) The functions k + and k − are C 1 and there exists a constant C 3 > 0 such that
(2.9) (f) There exist a − ≤ a + such that the function is strictly decreasing and positive in − ∞, a − , is strictly negative inside a − ,a + , if a − < a + , is strictly increasing and positive in a + ,∞ . (2.10) A. Jack and M. Zervos 5 Note that Assumption 2.2(a) implies the nondegeneracy condition (ND) and the local integrability condition (LI) in [10, Section 5.5 of Karatzas and Shreve].It follows that, given an initial condition x ∈ R, the uncontrolled diffusion has a unique weak solution up to a possible explosion time.Moreover, the scale function and the speed measure that characterise a one-dimensional diffusion such as the one in (2.11), which are given by ) respectively, for any given choice of x 0 ∈ R, are well defined.At this point it is worth noting that the conditions in our assumptions do not involve a convexity assumption on h, often imposed in the stochastic control literature.Also, although they appear to be involved, it is straightforward to check whether a given choice of the problem data satisfies the conditions of Assumption 2.2.

The solution of the control problem
With regard to the general theory of stochastic control, the solution of the control problem formulated in Section 2 can be obtained by finding a, sufficiently smooth for an application of Itô's formula, function w and a constant λ satisfying the following Hamilton-Jacobi-Bellman (HJB) equation: If such a pair (w,λ) exists, then, subject to suitable technical conditions, we expect the following statements to be true.Given any initial condition x ∈ R, which reflects the fact that the optimal values of the performance criteria considered are independent of the system's initial condition.The set of all x ∈ R such that defines the part of the state space in which the controller should exert minimal effort by increasing the process ξ + so as to position and then reflect the state process at the closest boundary point of the set in the positive direction.Similarly, the set of all x ∈ R such that defines the part of the state space where the controller should exert minimal effort by increasing the process ξ − so as to position and then reflect the state process at the closest boundary point of the set in the negative direction.The interior of the set of all x ∈ R such that defines the part of the state space in which the controller should take no action.With regard to the control problems considered, we conjecture that an optimal strategy is characterised by two points, x − < x + , and takes a form that can be described as follows.The controller exerts minimal effort so as to keep the state process within [x − ,x + ].Accordingly, with the exception of a possible jump at time 0, the process ξ + is continuous and increases on the set of times when X t = x − in order to reflect the state process back into the no action region ]x − ,x + [.Similarly, the process ξ − can have a jump the size of x − x + at time 0 so as to reposition the state process from its initial value x to the boundary point x + , if x > x + , and then increases on the set of times when X t = x + .
Assuming that this strategy is indeed optimal, we need a system of appropriate equations to determine the free-boundary points x − , x + and the constant λ.To this end we conjecture that the so-called "smooth-pasting condition" holds, which, in the case of the singular control problem that we consider here, suggests that the function w should be C 2 , in particular, at the free boundary points x − and x + .We therefore look for a solution (w,λ) to the HJB equation (3.1) such that ) The four equations resulting from (3.6) and (3.8) for x = x − and for x = x + , respectively, suggest that we should consider a fourth parameter.To determine such a parameter, we observe that the strict positivity of k + , k − and the fact that w is continuous imply that w should have a local minimum inside ]x − ,x + [, denoted by x 0 , so that w (x 0 ) = 0.With regard to this observation, we note that the solution to the ODE (3.5) with initial condition w (x 0 ) = 0 is given by where p x0 and m x0 are the scale function and the speed measure of the uncontrolled diffusion (2.11), defined by (2.12) and (2.13), respectively.It follows that we need to determine A. Jack and M. Zervos 7 the four parameters x − < x 0 < x + and λ that solve the nonlinear system of equations where g is defined by The following result, the proof of which is developed in the Appendix, is concerned with showing that the heuristic considerations above indeed provide a solution to the HJB equation (3.1).Lemma 3.1.Suppose that Assumption 2.2 holds.The system of (3.10), where g is defined by (3.11), has a solution (x − ,x 0 ,x + ,λ) such that x − < x 0 < x + , and, if w is a function satisfying (3.9) inside the interval ]x − ,x + [ and is given by (3.6) and (3.8) in the complement of ]x − ,x + [, then w is C 2 and the pair (w,λ) is a solution to the HJB equation (3.1).
We can now prove the main result of the paper that concerns the optimisation of the ergodic expected criterion.theorem 3.2.Consider the stochastic control problem formulated in Section 2 that aims at the minimisation of the long-term average expected criterion defined by (2.2)- (2.3).Suppose that Assumption 2.2 holds, and let the constants x − ,x 0 ,x + ,λ be as in Lemma 3.1.Then, given any initial condition x ∈ R, ) and the points x − , x + determine the optimal strategy that has been discussed qualitatively above.
Proof.Throughout the proof, we fix the solution (w,λ) to the HJB equation (3.1) that is constructed in Lemma 3.1.We also fix an initial condition x ∈ R. Consider any admissible control C x ∈ Ꮿ x such that J E (C x ) < ∞.Using Itô's formula for general semimartingales, we obtain 8 Ergodic singular control the second equality following because ξ s = ξ + s − ξ − s and the jumps of X coincide with those of ξ.Here (ξ + ) c and (ξ − ) c are the continuous parts of the processes ξ + and ξ − , respectively.Now, given any time s ≥ 0, Using this observation and the definitions (1.4), we can see that (3.13) implies Since the pair (w,λ) satisfies the HJB equation (3.1), it follows that By construction, w is C 2 , w (x) = k − (x), for all x ≥ x + , and w (x) = −k + (x), for all x ≤ x − .Therefore, in view of (2.8) in Assumption 2.2, there exists a constant C 4 > 0 such that For such a choice of C 4 , (3.16) yields Now, with respect to the positivity of k + and k − , and Assumption 2.2(b), A. Jack and M. Zervos 9 These inequalities imply which proves that the stochastic integral in (3.18) is a square integrable martingale, and therefore has zero expectation.In view of this observation, we can take expectations in (3.18) and divide by T to obtain In view of (3.21) and the definition of I T (C x ) in (3.15), we can pass to the limit T → ∞ to obtain J E (C x ) ≥ λ.
To prove the reverse inequality, suppose that we can find a control Plainly, (3.26) implies that X is nonexplosive, so that τ = ∞, P-a.s.Also, with regard to the construction of w, we can see that, for such a choice of a control, (3.15) implies Now, (2.5) in Assumption 2.2, (3.17) and (3.26) imply which proves that the stochastic integral in (3.29) is a square integrable martingale, and which proves that J E ( C x ) = λ, and establishes (3.12).Finally, we note that a control C x satisfying (3.26)-(3.28),which is optimal, can be constructed as in [16].
The following result is concerned with the solution to the optimisation problem considered with the ergodic pathwise criterion.theorem 3.3.Consider the stochastic control problem formulated in Section 2 that aims at the minimisation of the long-term average pathwise criterion defined by (2.4).Suppose that Assumption 2.2 holds, and let the constants x − , x 0 , x + , λ be as in Lemma 3.1.Then, given any initial condition x ∈ R, and the points x − , x + determine the optimal strategy.
Proof.Throughout the proof, we fix the solution (w,λ) to the HJB equation (3.1) that is constructed in Lemma 3.1.We also fix an initial condition x ∈ R. Consider any admissible control C x ∈ Ꮿ x .Using the same arguments as the ones that established (3.18) in the proof of Theorem 3.2 above, we can show that where In view of (2.5) in Assumption 2.2(a) and the second estimate in (3.17), it follows that where M is the quadratic variation process of the local martingale M defined in (3.35).Now, with regard to the Dambis, Dubins, and Schwarz theorem (e.g., see Revuz and Yor [14, Theorem V.1.7]),there exists a standard, one-dimensional Brownian motion B defined on a possible extension of (Ω,Ᏺ,P) such that In view of this representation, the observation that (3.39), and the fact that lim T→∞ B T /T = 0, we can see that lim However, in light of these inequalities and an argument such as the one establishing (3.42) and (3.43) above, we can see that J P ( C x ) ≡ lim T→∞ I T ( C x )/T = λ, and the proof is complete.