Optimal Oil-Owner Behavior in Piecewise Deterministic Models

Six simple piecewise deterministic oil production models from economics are discussed by using solution tools that are available in the theory of piecewise deterministic optimal control problems.


Introduction
Six simple piecewise deterministic models for optimal oilowner behavior are presented. Their central property is sudden jumps in states. The aim of this paper is to show in admittedly exceedingly simple models how available tools for piecewise deterministic models, namely, the HJB equation and the maximum principle, can be used to solve these models analytically. We are looking for solutions given by explicit formulas. That can only be obtained if the models are simple enough. The models may be too simple to be of much interest in themselves, but they can provide some intuition about features optimal solutions may have in more complicated models. Piecewise deterministic models have been used a number of times in economic problems in the literature; some few scattered references are given that contain such applications [1][2][3][4]. I have not been able to find references directly concerned with piecewise deterministic oil production problems. For different probability structures, and for discrete time, a host of related problems has been discussed in the literature; references to such literature have been left out, with one exception. Problems of control of jump diffusions, see [5], encompass piecewise deterministic problems, and some problems appearing in [5] are related to the ones discussed below. A classic reference to piecewise deterministic control problems is [2].
In all models below, an unbounded number of jumps in the state can occur at times 0 < 1 < 2 < 3 < ⋅ ⋅ ⋅ , and, when is given, +1 is exponentially distributed in [ , ∞) (all − −1 independent). Sometimes, the size of the jumps is influenced by stochastic variables . Let = ( 1 , 1 , 2 , 2 , . . .). At time , we imagine that the control values chosen can be allowed to depend on what has happened, that is, on , for < , but not on future events, that is, , for which > . Such controls (written ( , )) are called nonanticipative. Corresponding state solutions denoted by ( , ) are then also nonanticipative. (A general set-up, with further explanations, is given in Appendix.) Frequently below, ( , ) will be the wealth of the oil owner. In infinite horizon economic models, the weakest terminal condition that is natural to apply is an expectational no-Ponzigame condition; namely, lim inf → ∞ [ − ( , )] ≥ 0, where is the discount factor. Note that some stronger conditions will be used in some of the models presented in the sequel.
The interpretation of the model is that is consumption rate, is the size (in dollars) of an oil field found at time , is wealth, and is interest. Oil fields are sold immediately after discovery.
Let us solve problem (1)-(3) by using the extremal method (see the appendix). Now, with = − / + ( − ), solving the first-order condition for maximum of , we get that = /( −1) 1/( −1) maximizes the Hamiltonian. The we guess that we do not need the fact that ( , ) depends on arbitrary initial points ( , ), as in appendix. Equation (4) is satisfied by ( ) =̂− ,̂:= (we try the possibility that ( , ) is even independent of ). Now, for :=̂1 /( −1) , := ( − )/( −1) > 0, we get ( ) = /( −1) ( ( )) 1/( −1) = − . Because is concave in ( , ), is linear, and is independent of , sufficient conditions based on concavity (see Theorem 3 in Appendix) then give us that the control = − is an optimal control for problem (1)- (3). The optimal control is independent of and the 's. Write To see this, finḋ * ( ) and simply check that the differential equation is satisfied. Moreover, evidently * (⋅) satisfies * ( +) = * ( −)+ . (To ensure that * ( ) ≥ 0 a.s., we could have postulated at the outset that is so large that 0 ≥ .) We are now going to replace the bequest function ( ) by the terminal constraint ( , ) ≥ 0. For this purpose, we are now going to vary and hence alsôand . For = , consider the expectation of Φ ( , ). We will not give an explicit formula for this expectation, but we mention that it can be calculated in two steps, first given that jumps have happened in [0, ] and then the expectation with being stochastic (the rule of double expectations is then used). The expectation of the sum containing as well as 0 is positive, and the term in front of is negative. There thus exists a unique positive value of and, hence, of (denoted by ) such that Φ ( , ) = 0; that is, * ( , ) = 0.
If we drop the term ( ) in the criterion but add the terminal condition ( , ) ≥ 0, the free end optimality obtained in the original problem for = evidently means optimality of = − =̂1 /( −1) − = ( ) 1/( −1) − in the end constrained problem (the found value appears in , though now not in the criterion). (Alternatively, we could use the sufficient conditions for end constrained problems in Theorem 3 in Appendix to obtain optimality in the present end constrained problem. This would require us to check condition (A.13), which is easily done.) Let us discuss a little more the value of for large values of . Now, And, continuing, we get Denote by ∞ the value of for which Φ ∞ (∞, ) = 0, and define ∞ : , so is close to ∞ and := ( + ) is close to ∞ . So for being large, the optimal control is approximately ∞ − . Note that belongs to ( − , ∞) and both and ∞ are increasing in ∈ (0, 1), so a larger means we consume more in the beginning. In the problem where ( ) is replaced by the terminal constraint ( , ) ≥ 0, the optimal control depends on , . It is immediately seen that ∞ is increasing in and in each , so also ∞ has these properties, indicating, for being large, that even and have these properties. This conclusion actually follows for any , because [∑ =1 − ] is evidently increasing in and in each . We may assume that the jumps are stochastic, that is, that ( +) − ( −) = , where { } take values in a common bounded set and are independent, and are independent of the 's. If we then assume that ∑ ( ) < ∞, the solution in this problem is the same as the one above for = ( ).
Note that we must assume that we have a deal with the bank in which our wealth is placed that it accepts the above behavior. That is, before time 0, we have got an acceptance for the possibility of operating with this type of admissible solutions, which means that only in expectation we leave a wealth in the bank ≥ 0. In actual runs, sometimes we leave in the bank a positive wealth (that the bank gets), and for other runs a negative wealth (debt) that the bank has to cover.
Consider now the case where = ∞, = 0 in (1). Then, Journal of Calculus of Variations 3 when = ∞ . In this case, = ∞ − is optimal in the problem where lim inf → ∞ [ ( ) ( , )] ≥ 0 is added as an end constraint. (See the appendix, (A.14).) The latter condition is equivalent to the so-called no-Ponzi-game condition.

Model 2
(This model Is related to exercise 4.1 in Øksendal and Sulem [5].) Consider the following problem: , where , are fixed positive constants where is the control, subject tȯ = − , , 0 are given positive numbers, where is a given positive number, > , , = 1, 2, . . ., is exponentially distributed in ( −1 , ∞) with intensity and is decreasing towards zero ( 0 = 0, all − −1 independent). In contrast to problem (1)-(3), now the jumps are linearly dependent on ( −). To defend such a feature, one might argue that the richer we are, the more we are able to generate large jumps (the jump may actually represent a collection of oil finds). On the other side, we will assume that such jumps occur with smaller and smaller intensities.

Journal of Calculus of Variations
In fact, as is easily seen, this holds for all : * < * +1 .
Let * ( , ) be the solution for ≡ 0. By (11), * ( ) Choose the smallest * such that * < − . If * jumps occur at once at 0 ( = ∞, ≤ * ), while further jumps occur with intensity = * , then Γ would Hence, Γ ≤ (1 + ) * * . For any admissible solution ( , ), ( , ) ≤ * ( , ), for all ( , ); hence, for in the appendix is satisfied. Hence, sufficient conditions hold (see Theorem 2 in Appendix) and =̃. (In the appendix, for the sufficiency of the HJB equation to hold, it is required that * ( , ) is bounded if < ∞ (for = ∞, we need boundedness for in all bounded intervals). This is not the case here. But we could have replaced by V , V being the control. Assume that we require V( , ) to be bounded in the above manner. Now, V * ( , ) is so bounded, and it is then optimal in the set of such bounded V(⋅, ⋅)'s. ) We can show that the 's are increasing in and in each , > . The simplest argument is that this must be so, when we now know that is the optimal value function after jumps at (before) = 0.

Model 3
In this model, the physical volume of oil production is constant = 1, but the oil price jumps up and down at Poisson distributed time points , = 1, 2, . . .. Let 0 = 0. Consider the problem given positive constants, where is the control, subject to the differential equations below and where is taking a finite number of values { } with given probabilities , all are identically distributed, and is exponentially distributed in ( −1 , ∞) with intensity (all the random variables − −1 , independent). Let V = . The two states and are governed by (18) (no jumps in ) anḋ (0) = 0 , (0) = 0 , where 0 and 0 are given constants; in fact, it will be convenient to assume that 0 = V. The end constraint [ ( )] ≥ 0 is required to hold. Here, is wealth and is interest earned.
Let us use the extremal method (see the appendix) for solving this problem. In what follows, it is guessed that adjoint functions do not depend on arbitrarily given initial points (which in the appendix are denoted by ( , )). The adjoint function corresponding to is hence denoted by ( , ). We make the guess that ( , ) simply equals − (independent of ), being an unknown. It does satisfy the adjoint equatioṅ Next, maximum of the Hamiltonian = − / + ( + − ) is obtained for satisfying the first order where = 0 − /( + ).

Model 5
Consider the problem where is a fixed positive number, where is a fixed positive number, (0) = 0,̇= 0, (28) and = 1− /(1 − ), where > 0, are given constants, ∈ (0, 1), ≥ 0 is the control, and is the intensity of jumps. All fields found are of the same size . We imagine that fields are sold immediately when they are found or that they are produced over a fixed period of time, with a fixed production profile. In both cases, we let income from a field discounted back to the time of discovery be equal to 0 := , for a given > 0. The oil owner wants to maximize the sum of discounted incomes earned over [0, ∞).
Let us try the proposal that the current value optimal value function = , where , an unknown. The current value HJB equation is which implies that the maximizing satisfies 0 − = 1, Trivially, (A.7) in the appendix holds, so the sufficient conditions in Theorem 2 are satisfied and = ( 0 ) 1/ is optimal. Because > 0, in expectation (but not in all runs), over [0, ∞), a positive sum of discounted incomes (= ) has been earned.

Model 6
Consider the problem where is a fixed positive number, no jumps in , = , , are fixed positive numbers, < 1, is the intensity of jumps in . All fields found are of the same size . We again imagine that fields are sold immediately when they are found or that they are produced over a fixed period of time, with a fixed production profile. Let income from a field discounted back to the time of discovery be equal to 0 := , for a given > 0. The state ( ) is the amount of expertise available for finding new fields, built up over time according tȯ= 1− /(1 − ), where is money per unit of time spent on building expertise.
Let us try the proposal that the current value function equals + , and being unknowns. The current value HJB equation is

Comparisons
In Models 1, 2, 5, and 6, oil finds are made at stochastic points in time; in Models 3 and 4, it is the price of oil that changes at stochastic points in time. In Model 1, we operate with the constraint (∞, ) ≥ 0 (for = ∞, = 0), where is the oil-owners' wealth. Here, for some runs, (∞, ) can be negative ( > 0 ) and, for other runs, positive. In Model 2, we required ( , ) > 0 for all , all . (The results in that model would have been the same if we had required only (∞, ) ≥ 0 for all .) In Model 2, the optimal control comes out as stochastic and not deterministic as in Model 1. Moreover, as a comment in Model 2 says, as a function of time, the optimal control decreases more rapidly in Model 1 as compared to Model 2. The latter feature stems from the fact that, in Model 2, we enhance future income prospects by not decreasing too fast, because the jump term (the right-hand side of the jump equation) depends positively on , which is not the case in Model 1.
In Models 3 and 4, the oil price exhibits sudden stochastic jumps. In Model 3, the rate of oil production is constant, but income earned (as well as interest) is placed in a bank after subtraction of consumption. In Model 4, income earned, after subtraction of consumption, is reinvested in the oil firm to increase production. In Model 4, the optimal control is stochastic; it depends on whether the current price is high or low. In Model 3, the control is deterministic, and it depends only on the expectation V of the stochastic price. Consider the case where, in Model 3, the expected price V is zero and In the extremely simple Models 5 and 6, the frequency of oil finds is not fixed but influenced by a control. In Model 5, the current frequency (or intensity) is determined by how much money is put into search at that moment in time.
In the simplest case considered in Model 5, a find today does not influence the possibility of making equally sized discoveries tomorrow. Then it, is not unreasonable that the optimal control (which equals ( 0 ) 1/ ) is independent of the discount rate but dependent on the fixed value of the finds 0 = . (Here, actually could be the expected size of a find, in both Model 5 and Model 6; we could have had the sizes of the finds being independently stochastic, with being the expected value of the sizes.) In Model 6, it is the ability to discover oil that is built up over time, so with greater impatience (higher ), we should expect less willingness of devoting money to increase this ability, and this shows up in the formula = ( 0 / ) 1/ . See pages 147 and 155 (and for a proof, see page 168) in [6]. (Formally, we only need to assume that the nonanticipative function * ( , ) is measurable and bounded.)

Appendix
In case of restrictions of the form ( , ) ∈ ( ), we must assume that * ( , ) satisfies these restrictions, in this case for all admissible (⋅, ⋅).
(This is the infinite horizon current value form of (A.4); compare (3.71) and (3.69), page 150 in [6].) The Extremal Method. The so-called extremal method (see page 117 and pages 126-130 in [6]) now to be described yields solutions satisfying a maximum principle (necessary condition). Now, it is assumed that sup < ∞ and that does not depend on ( , , ). To the assumptions above on 0 , , 0 , , and ℎ 0 , we now add the assumption that these five functions have derivatives with respect to that are continuous in ( , , , ). Let be the identity matrix and let = 0 + . For arbitrary ( , ), we seek controls