MONOTONICITY AND DIFFERENTIAL PROPERTIES OF THE VALUE FUNCTIONS IN OPTIMAL CONTROL

Using the “basic monotonicity property” along locally admissible trajectories, we extend to very general problems certain existing results concerning the differential inequalities verified by the value function of an optimal control problem; these differential inequalities are expressed in terms of its contingent, quasitangent, and peritangent (Clarke’s) directional derivatives and in terms of certain sets of “generalized tangent directions” to the “locally admissible trajectories.” Under additional, rather restrictive hypotheses on the data, which allow suitable estimates (and even exact characterizations) of the sets of generalized tangent directions to the trajectories, the differential inequalities are shown to imply previous results according to which the value function is a “generalized solution” (in the “contingent,” “viscosity,” or “Clarke” sense) of the associated Hamilton-Jacobi-Bellman (HJB) equation.


Introduction.
The aim of this paper is to extend to very general problems certain results in [3,4,5,7,8,10,11,13,14], concerning monotonicity and differential properties of the value functions in optimal control.
The main idea is, in the first place, to replace the so-called dynamic programming principle used by most of the authors, by the stronger monotonicity property along locally admissible trajectories of the value function.
Secondly, the monotonicity property is made more explicit in the form of certain contingent, quasitangent, and peritangent differential inequalities, using corresponding sets of generalized tangent directions to the locally admissible trajectories.
Finally, in the particular cases in [14,15,20] in which certain "lower estimates" of the sets of generalized tangent directions may be obtained, further, more explicit differential properties are obtained, including some well-known characterizations of the value functions as generalized solutions (in the viscosity, Clarke, or contingent sense) of the associated Hamilton-Jacobi-Bellman (HJB) equation; in particular, we prove in Corollary 6.3 the fact that under usual (rather restrictive) hypotheses on the data, the value function it is not only a "viscosity solution" but, in fact, a "strict viscosity subsolution" on its effective domain and, moreover, a "strict viscosity solution" on a certain "relative interior" of its domain.
For the sake of simplicity we consider here only autonomous (i.e., time-invariant) optimal control problems although the results in [13,14], and so forth show that the results in this paper may be naturally extended to nonautonomous problems.
The paper is organized as follows.In Sections 2 and 3, we recall the preliminary concepts and results from, respectively, nonsmooth analysis and the theory of the sets of generalized tangent directions to the locally admissible trajectories of some (constrained) differential inclusions with certain local regularity properties.In Section 4, for the sake of completeness, we present the detailed proof of Theorem 4.1 concerning the basic monotonicity properties of the value function along locally admissible trajectories.
In Section 5, we prove our first main result, Theorem 5.1, providing differential inequalities of the value function expressed in terms of the corresponding sets of generalized tangent directions to the locally admissible trajectories.In Section 6, we present, as a mere corollary, the second main result, Theorem 6.1, providing more explicit differential inequalities in some particular cases.
Finally, in Section 7 we consider in some detail an autonomous version of Example 1.4.5 in [5] to illustrate some of the theoretical aspects in the previous sections.

Preliminary concepts and results from nonsmooth analysis.
As is well known from significant examples, the domain Y := Y 0 ∪ Y 1 ⊆ R n of the value function W (•) in Section 4 is, generally, neither an open subset of R n nor a differentiable (sub-) manifold and the value function itself is, generally, not differentiable (sometimes not even continuous); therefore, the monotonicity property (i) of Theorem 4.1 may be explicitly illustrated only using suitable concepts from nonsmooth analysis.
To make this paper as self-contained as possible, we recall very shortly the main concepts and results of this type to be used in the next sections; for other concepts and results of this type, we refer to [2,7,22], and so forth.
We recall first that the unilateral contingent, quasitangent, and Clarke (peritangent) cones to a subset X ⊆ R n at a point x ∈ X are the subsets (closed cones with vertex at 0) defined, respectively, by are related as follows: and only Clarke's (peritangent) cones C ± x X are always convex (closed) cones.If g(•) : X ⊆ R n → R is a real-valued function and its epigraph and subgraph (hypograph) are defined, respectively, by E(g) := (x, y) ∈ X × R; y ≥ g(x) , S(g) := (x, y) ∈ X × R; y ≤ g(x) , (2.3) then for each type of tangent cone, τ ∈ {K, Q, C}, one may define the corresponding unilateral τ-extreme directional derivatives at the point x ∈ X in direction v ∈ τ ± x X as follows: ) E(g) , (2.4) which, in view of the first properties in (2.2), are related as follows: Some of the results using nonsmooth analysis may be expressed in weaker forms using the corresponding τ-semidifferentials defined by which, due to the relations in (2.5), may be expressed also in terms of the left variants of the directional derivatives.Due to the inclusions in (2.2), the generalized contingent, quasitangent, and peritangent (Clarke) derivatives in (2.4), (2.6) for τ ∈ {K, Q, C}, respectively, are, obviously, related as follows: (2.7) and simple examples show that each of these inequalities and inclusions may be a strict one.
From the definitions in (2.1) of the contingent, quasitangent, and Clarke (peritangent) cones, one may derive more "explicit" equivalent definitions of the generalized derivatives in (2.4), (2.6); the simplest ones are the following usual definitions of the extreme contingent derivatives in (2.4): which coincide with the usual (Fréchet) derivative at x ∈ Int(X) whenever the latter exists.In particular, for real functions ω(•) : I = [a, b] ⊂ R → R, the extreme contingent derivatives in (2.9) in direction 1 ∈ K ± t I coincide with the well-known Dini derivatives and with the usual derivatives, ω (t), whenever the latter exist.
On the other hand, as shown in [22], the quasitangent and peritangent extreme directional derivatives in (2.4) allow much more complicated equivalent definitions using certain "inf-sup mixed limits"; however, in the very particular case in which g(•) is locally Lipschitz at the interior point x ∈ Int(X), the peritangent directional derivatives in (2.4) coincide with the well-known Clarke generalized directional derivative p, v , (2.11) where is the well-known Clarke generalized gradient in (2.6).Further, for vector-valued mappings and, more generally, for set-valued mappings F(•) : X ⊆ R n → ᏼ(R m ) (where ᏼ(R m ) denotes the family of all subsets of R m ), for each type of tangent cone, τ ∈ {K, Q, C}, one may define corresponding one-sided set-valued τ-directional derivatives at a point x X by the general formula which, in view of the definitions in (2.1), may be more explicitly characterized in each of the cases τ ∈ {K, Q, C}.
In particular, for the vector-valued mappings x(•) : I = [a, b] → X, we will use the set-valued contingent derivatives in direction r ∈ {1, 0 + } defined by which, in the case r = 1, coincide with the usual derivatives, x (t), whenever the latter exist, while in the case r = 0 + they coincide with the "horizon" contingent derivatives K ± x(t;0 + ) ⊂ R n which is a cone with vertex at 0 ∈ R n .

Generalized tangent directions to the locally admissible trajectories.
In this section, we recall some of the results in [14,16,20] concerning lower estimates of certain sets of generalized tangent directions to the absolutely continuous (AC) trajectories in F (y; Y 0 ) of the constrained differential inclusion defined by the "orientor field" and by its domain, the subset Y 0 ⊆ R n ; here, the interval I(x(•)) ⊂ R is one of the following forms: (−τ 1 ,t 1 ), [0,t 1 ), (−τ 1 , 0], for some t 1 ,τ 1 > 0. In particular, the multifunction F(•) may be of the continuously parameterized type where U is a nonempty set and f (•,u), u ∈ U , are continuous mappings; in this case a solution x(•) of (3.1) satisfies for a suitable control mapping u(•) : From the multitude of possible distinct sets of tangent and generalized tangent directions to the trajectories of (3.1) considered in [16,20], we will consider in the first place the most natural ones, defined, respectively, as the sets of usual tangent directions, of contingent directions, and of horizon contingent directions: However, in the problems to be studied in the next sections, one may use larger sets of generalized tangent directions that may be defined using the contingent, quasitangent, and peritangent cones in (2.1) to the reachability set, F := {(t, y, x(t)); t ∈ R, y ∈ Y 0 , x(•) ∈ F (y; Y 0 )}, which is the graph of the reachability multifunction, (t, y) R F (t, y) := {x(t); x(•) ∈ F (y; Y 0 )}, associated to the problem in (3.1).Therefore, as in [16,20], we introduce the following sets of generalized contingent, quasitangent, and peritangent directions and also the horizon generalized contingent and peritangent directions, defined, respectively, by which, obviously, are related as follows: As upper estimates of the sets of contingent directions above, one may take the relations which are proved in [16,Theorem 3.3], for arbitrary orientor fields, using the wellknown mean value theorem for the Lebesgue integral (see, e.g., Aubin and Cellina [1]); moreover, if the orientor field F(•) has the affine growth property in the sense that there exists M > 0 such that then the inclusions in (3.7) may be "refined" to the following ones: Noting that the upper estimates in (3.7), (3.9) are useful for sufficient conditions of various types, it is obvious that for necessary conditions one needs lower estimates and possibly exact characterizations of the sets of generalized tangent directions in (3.4)- (3.5).
In view of applications to optimal control and differential games and due to the local nature of the concepts in (3.4)- (3.5), in what follows we will assume that the orientor field F(•) in (3.1) satisfies one of the following hypotheses at a given point y ∈ Y 0 .
and the restriction F(•)|B r (y) is "continuously parameterized" in the sense that it is given by (3.2) and the mappings f (•, u), u ∈ U , are continuous.
We summarize the results in [6,9,14,20] in the following result giving lower estimates of the sets of generalized tangent directions in (3.4)-(3.5).

Theorem 3.4. (i) If Hypothesis 3.1 is satisfied, then
where the cone of unbounded (horizon) directions of an (unbounded) subset A ⊂ R n is defined by (iii) If Hypothesis 3.3 is satisfied, then the property in (3.11) holds and (3.12) may be refined to the following: Remark 3.5.We note that while the relations in (3.11), (3.14) for the continuously parameterized orientor fields in (3.2)-(3.3)are proved in [14] using a certain refinement of Peano's theorem for bilateral solutions, x(•) : (−t 1 ,t 1 ) → Y 0 of (3.1), (3.3), the relations in (3.10), (3.11), and (3.12) for nonparameterized differential inclusions are proved "directly" only for the right-hand side solutions, x(•) : [0,t 1 ] → Y 0 , of (3.1), and therefore, only for the right-hand side tangent directions, T + F (y), K + F (y), and so forth, since the basic results in the theory of differential inclusions (see, e.g., Filippov [9]) are of this type; however, the simple trick of considering the associated orientor field, G(y) := F(−y), y ∈ −dom(F (•)), which has the same regularity properties and, moreover, and so forth, allows the extension to the left-hand side solutions of any result concerning the "right-hand side" ones.
Remark 3.6.In the absence of Hypotheses 3.1, 3.2, and 3.3, in particular at the boundary points, y ∈ ∂Y 0 := Cl(Y 0 ) \ Int(Y 0 ), it seems very hard to obtain general results of the type in Theorem 3.4; however, in many particular cases one may use ad hoc arguments to obtain reasonable lower estimates of the sets of generalized tangent directions in (3.5) even in the case where none of the Hypotheses 3.1, 3.2, and 3.3 is satisfied; these types of results are particularly needed in the theory of differential games (see, e.g., Miricȃ [17,19]) where certain irregular feedback strategies define discontinuous orientor fields which, however, may verify one of the Hypotheses 3.1, 3.2, and 3.3 at some initial points y ∈ Y 0 .

Monotonicity properties of the value function.
In what follows we will consider a Bolza autonomous optimal control problem for differential inclusions, over a prescribed set of admissible trajectories, Ω α (y), y ∈ Y 0 , defined as AC mappings x(•) that satisfy constraints of the form As is apparent from this succinct formulation, for each initial point y ∈ Y 0 and for each the admissible trajectory x(•) ∈ Ω α (y), the terminal time t 1 = t 1 (y; x(•)) > 0 is free and it may be interpreted as the first moment at which the last two conditions in (4.2) (that define the terminating rule of the process) are verified; in what follows we assume that Y 0 ∩ Y 1 = ∅ to avoid possible ambiguities; one may also note that problem Ꮾ A denotes in fact the family of optimization problems Ꮾ A (y), y ∈ Y 0 , defined by (4.1)-(4.2).Moreover, as in most cases, the class Ω α of admissible trajectories is either the largest class Ω 1 = AC of absolutely continuous mappings or one of the remarkable subclasses: Ω ∞ of Lipschitzian trajectories, Ω r of regular trajectories, and Ω pc of piecewise smooth trajectories (with piecewise continuous derivatives).
We note that the results to follow remain valid for the more particular classes of parameterized (standard) autonomous optimal control problems P over a prescribed set of admissible pairs (x(•), u(•)) ∈ (Ω α × ᐁ α )(y), y ∈ Y 0 , that satisfy constraints of the form (4.2) and and where the sets Ω α (y), y ∈ Y 0 , of admissible trajectories are "generated" by a corresponding class, ᐁ α , of admissible controls.
In the dynamic programming approach, many authors are considering only the proper value function while the extended (full) value function (see, e.g., Cesari [5], etc.), proved to be more efficient in certain developments of the theory.
For the sake of completeness we provide here the detailed proof of the following result which may be considered as a refinement of Propositions 4.5.(i),4.5.(ii) in Cesari [5]; the progress consists in the use of the locally admissible trajectories in Ω loc α (y) := S α F (y; Y 0 ) ⊃ Ω α (y), y ∈ Y 0 , which allow the use of the concepts and results in Section 3; one may note that an admissible trajectory x(•) ∈ Ω α (y) in (4.2) for the problem Ꮾ A may be considered as a locally admissible trajectory in Ω loc α (y) in (3.1) for which I(x(•)) = [0,t 1 ), and there exists x(t 1 ) ∈ Y 1 while, for a locally admissible one, either x(t 1 ) may not exist, or it may not belong to the terminal set Y 1 ; in what follows, for each y ∈ Y 0 we denote by Ω α (y) := { x(•) ∈ Ω α (y); Ꮿ(y; x(•)) = W 0 (y)} the (possibly empty) set of the optimal trajectories corresponding to the initial point y ∈ Y 0 .
In the remaining case, 0 ≤ s 1 < s 2 < t 1 , we note first that if Ω α (x(s 2 )) = ∅, then according to the natural convention inf ∅ = +∞ one has ω is an admissible trajectory for the point x(s 1 ) ∈ Y 0 ; therefore as before, adding the integral s 1 0 g 0 (x(t), x (t))dt to both sides of this inequality we obtain ω x (s 1 ) ≤ ω x (s 2 ) and statement (i) is proved in the case x(•) ∈ Ω loc α (y) is a locally admissible trajectory and s 1 ,s 2 ∈ I(x(•)) are such that 0 ≤ s 1 < s 2 .In the remaining case, in which s 1 < 0, the proof is practically the same replacing y by y 1 := x(s 1 ) ∈ Y 0 and the trajectory x(•) by the "shifted" one, x 1 (t) := x(t +s 1 ), t ∈ [0,t 1 −s 1 ), t 1 ≥ s 2 , which is locally admissible for y 1 .
(ii) According to a well-known result in analysis, the monotonic function ω x (•) in (4.7) is regulated, hence from the fact that it is increasing it follows that there exists lim t t 1 ω x (t) ≤ ω x (t 1 ) = Ꮿ(x(•)).On the other hand, from conditions in (4.2) it follows that the function x 0 (•) is absolutely continuous, hence from (4.3) it follows that Using the well-known Lebesgue theorem according to which a monotonic function is differentiable a.e. , from Theorem 4.1 one obtains the following abstract necessary properties of the value function.
One may note here that the abstract character of the properties in (4.13), (4.14) is given by the fact that these conditions are not expressed in terms of the data F(•), g 0 (•, •), W (•) of the problem.

The main results.
The main results of this paper concern certain differential inequalities verified by the proper value function, W 0 (•), and also by the extended value function, W (•), in (4.6); since for some initial points y ∈ Y 0 there may not exist admissible trajectories (in which case we should take W 0 (y) := inf ∅ := +∞) and at some other initial points one may have W 0 (y) = −∞, the given set Y 0 ⊂ R n of initial points may be "partitioned" as follows: (5.1) moreover, the effective domain, Y d 0 = dom(W 0 (•)) ⊆ Y 0 , of the proper value function, may be further partitioned as follows: which means that for the initial points y ∈ Y i 0 := Y d 0 \ Y 0 , the cost functional Ꮿ(y; •) in (4.1) has a finite infimum which it is not a minimum.
On the other hand, among the initial points y ∈ Y 0 (that have optimal trajectories) one may distinguish those for which the optimal trajectories may be "continued" backwardly, that is, the points in the subset this subset may be considered a kind of "relative interior" of the subset Y 0 since if z ∈ Y 0 , x(•) ∈ Ω α (z), then x(s) ∈ Y − 0 for all s ∈ (0, t 1 ); consequently, the "complementary" subset Y 0 \ Y − 0 may be interpreted as a "relative boundary" of Y 0 since its points are dead ends of the corresponding optimal trajectories.
Similarly, the set of terminal points, Y 1 ⊂ ∂Y 0 , may be partitioned as follows: meaning that the points ξ ∈ Y ∞ 1 cannot be reached by any admissible trajectory while the points ξ ∈ Y −∞ 1 are reached by some admissible trajectories but not by optimal ones; therefore the subset Y ∞ 1 ⊂ Y 1 may be deleted from the set Y 1 of terminal points without modifying the problem.
In view of the fact that the extended trajectory x(•) 2) is a solution of the extended differential inclusion most of the results in what follows will be expressed in terms of the associated extended orientor field F(•).
However, since F(•) is of a very particular type and any trajectory x(•) = (x(•), x 0 (•)) ∈ F ( y) of (5.5) is perfectly determined by the trajectory x(•) ∈ F (y) of the inclusion in (3.1), and by x 0 (•) given by the formula in (4.2), the corresponding sets of generalized tangent directions in (3.4)-(3.5)may be described as follows: where Obviously, Theorem 3.4 remains valid for the extended orientor field F(•) in (5.5) and for the corresponding sets of generalized tangent directions in (5.6).However, this result may not be true anymore for the sets of left tangent directions, K − F (ξ), GK − F (ξ), P − F (ξ), K −∞ F (ξ), and so forth, defined in the same way as in (5.6) at the terminal points since ξ ∈ ∂Y 0 is a boundary point.We recall that, as in Section 3, in the case where the extended orientor field F(•) in (5.5) has the affine growth property in (3.8), the horizon tangent directions in (5.6) may simply be ignored since in this case The first main result of this paper is the following generalization of [14, Theorem 5] which has been proved for nonautonomous continuously parameterized optimal control problems of Mayer type.
, the "upper" peritangent and contingent derivatives, respectively, of the proper value function satisfy the following inequalities:

the extended value function satisfies similar inequalities for the left tangent directions:
(5.10) (5.11) and at each optimal terminal point ξ ∈ Y 1 (reached by an optimal trajectory), the lower left contingent derivatives in (2.9) satisfy the inequalities (5.12) (iii) At each initial point y ∈ Y 0 (which has an optimal trajectory), the lower contingent derivatives of the proper value function satisfy

continuous function, and F(•) has the affine growth property in (3.8), then at each initial point y ∈ Y i 0 (at which one has an finite infimum but not an optimal trajectory), the proper value function satisfies the inequalities
and note that in view of the definitions in (2.4), (2.1) the corresponding inequality in (5.8), is obviously equivalent with the relation where E(W 0 ) denotes the epigraph in (2.3).
To prove the last property, in view of the definition in (2.1), we consider an arbitrary sequence (y m ,z m ,t m ) → (y, W 0 (y), 0 − ) such that (y m ,z m ) ∈ E(W 0 ), hence such that z m ≥ W 0 (y m ) for all m ∈ N.
In the first case, in which v ∈ P − F (y), from (5.6) it follows that for (y m ,t m ) → (y, 0 − ) there exist x m (•) ∈ F (y m ,Y 0 ), m ∈ N, such that Next, since t m < 0, from the monotonicity property (i) in Theorem 4.1 it follows that m for all m ∈ N, the inequalities in (5.19) imply the fact that (y m +t m v m ,z m −t m v 0 m ) ∈ E(W 0 ) for all m ∈ N, which proves (5.17), hence also (5.16).
In the second case, in which v ∈ P −∞ F (y), the proof of (5.16) is very similar: from (5.6) it follows that for (y m ,t m ) → (y, 0 − ) there exist r m → 0 + , x m (•) ∈ F (y m ,Y 0 ) such that (5.20) Next, since t m r m < 0, from the monotonicity property (i) in Theorem 4.1 it follows that for all m ∈ N, which proves (5.17), hence also (5.16); statement (i) is proved since the analogous inequalities in (5.8) for v ∈ P + F (y) ∪ P ∞ F (y) follow in the same way using the subgraph S(W 0 ) in (2.3).
On the other hand, due to the more explicit characterizations in (2.9) of the extreme contingent derivatives, the proof of the inequalities in (5.9) is much easier.
To make a choice, let y ∈ Y d 0 , v = (v, v 0 ) ∈ GK + F (y) and, according to the definitions in (5.6), let t m → 0 + and x m (•) ∈ F (y; Y 0 ) be such that Since t m > 0 for all m ∈ N, from Theorem 4.1 it follows that ω xm (t m ) := W 0 (x m (t m ))+ x 0 m (t m ) ≥ ω xm (0) = W 0 (y) hence in view of (2.9) we may write successively and (5.9) is proved in this case; since in the other cases, v ∈ GK − F (y) ∪ GK ±∞ F (y), (5.9) follows in a very similar way, statement (i) is proved.
To prove the inequality in (5.12), we consider ξ ∈ Y 1 and note that according to the definition in (5.4), there exist y ∈ Y 0 and an optimal trajectory x(•) ∈ Ω α (y) such that x( t 1 ) = ξ; according to Theorem 4.1, the real function ω x (•) in (4.7) is constant, hence In what follows we will consider separately the following two complementary cases: (I) and since from (5.24) it follows that from the characterizations in (2.9) of the extreme contingent derivatives it follows successively that which proves (5.12) in case (I).
(iii) The proof of the inequalities in (5.13)-(5.14) is entirely similar to the one above of (5.12) so we may omit it.
(iv) If y ∈ Y i 0 := Y d 0 \ Y 0 is an initial point at which the functional Ꮿ(y; •) in (3.1) has a (finite) infimum which is not a minimum, then, as is well known, for each m ∈ N there exists an admissible trajectory x m (•) ∈ Ω α (y) such that (5.30) next, from the monotonicity property in Theorem 4. (5.31) We will prove now the fact that the affine growth property in (3.8) implies the following relation verified by any admissible trajectory x(•) ∈ Ω α (y):

.38)
We take now t m := (1/ √ m) → 0 + and note that there exists m 1 ≥ m 0 such that t m ∈ (0,t 1 m ) for all m ≥ m 1 , hence according to (5.31) one has (5.39) As in the proof of statement (ii) we consider the two complementary cases (I) (5.41) In case (I), to simplify the notations we may assume that y), hence from (5.39), as in the previous cases, (5.15) follows.In case (II) we note that from (5.32) it follows that s m := x m (t m )− y → 0 + and also that r m := t m /s m → 0 + ; next, since the sequence {( x(t m ) − y)/s m ; m ∈ N} is bounded, it has a convergent subsequence, hence we may assume that ; finally, from (5.39) it follows that which, as in the previous cases, implies (5.15) and the theorem is proved.
Remark 5.2.The hypothesis in statement (iv) according to which F(•) has the affine growth property in (3.8), though quite frequent in optimal control theory (see, e.g., Berkovitz [4], Cesari [5], etc.), is rather restrictive and Example 7.1 shows that it may be relaxed; however, the rather weak interest of the initial points y ∈ Y i 0 = Y d 0 \ Y 0 at which the cost functional has a finite infimum but not a minimum does not seem to justify the efforts in this direction.
Due to the relations in (2.7), the inequalities in (5.8) imply the corresponding ones in (5.9) but only on the subsets of peritangent directions P ± F (y)∪ P ±∞ F (y); on the other hand, due to the inclusions in (3.6), the inequalities in (5.13) are stronger than those in (5.15).Remark 5.3.We recall that Berkovitz [4] introduced the following lower tangential derivative: which may be interpreted as a restriction of the lower right contingent derivative in (2.9) to the graph of the reachability multifunction F (•; •); as is easy to see, one has •) is a rather "abstract" concept since F (•; •) may rarely be found in explicit form.
On the other hand, the same proofs as those of (5.11), (5.13) lead, obviously, to the following (abstract) refinements: which, in particular, imply the following relation: an obvious generalization of the result in [4,Theorem 3.1].We note also that in the (frequent) case in which W 0 (•) is locally Lipschitz at the point y ∈ Y d 0 , using the fact that in this case one has for all m ∈ N, one may easily obtain the following inequality: which, together with (5.13)-(5.14),leads to the relations which are not only stronger but also less abstract than the relations in (5.46).

Further properties under restrictive hypotheses.
The results in Theorem 5.1 above still have an abstract character since they are not expressed in terms of the data, F(•), g 0 (•, •), of the problem; however, these results allow more explicit variants as soon as suitable lower estimates of the corresponding sets of generalized tangent directions in (5.6) are obtained.The most general cases in which such estimates are available are those in Theorem 3.4; however, as experience shows (see, e.g., Example 7.1), reasonable lower estimates may be obtained in many other particular cases, including at the "boundary", terminal points, ξ ∈ Y 1 ∪ Y −∞ 1 .From Theorem 5.1 and Theorem 3.4 we obtain the following result providing more explicit differential properties of the value function for the types of problems that satisfy one of the Hypotheses 3.1, 3.2, and 3.3; these results will be expressed in terms of the values, F(y), of the extended orientor field in (5.5), of their closed convex hulls, co F(y), y ∈ Y 0 , and (when it is the case) of their sets of unbounded directions, D ∞ (co F(y)) in (3.13), at the points y ∈ Y d 0 at which the corresponding hypothesis is verified.

(i) If F(•) satisfies Hypothesis 3.3 at the initial point y
inf and if, in addition, F(•) is upper semicontinuous at y, then (ii) If F(•) satisfies Hypothesis 3.2 at the initial point y ∈ Y d 0 , then W 0 (•) satisfies (6.4), (6.5), and the following inequalities: (iii) If F(•) satisfies Hypothesis 3.1 at the initial point y ∈ Y d 0 , then W 0 (•) satisfies (6.4), (6.5), and and if, in addition, Ω α ⊆ Ω ∞ (in particular if F(•) is locally bounded), then (6.4), (6.5) may be replaced, respectively, by inf and if, in addition, Proof.Statements (i), (ii) and the first part of statement (iii) follow obviously from Theorem 5.1, from the lower estimates in Theorem 3.4 of the sets of generalized tangent directions in (5.6), from the upper estimates in (3.7), according to which K ± F (y) ⊆ F co (y), K ±∞ F (y) ⊆ F co ∞ (y), and also from the results in [16,20] according to which if F(•) is upper semicontinuous, in particular if it satisfies one of the Hypotheses 3.1, 3.2, then one has co F(y) = F co (y), D ∞ co F(y) = F co ∞ (y); (6.15) while in the general case one has co F(y) ⊆ F co (y), D ∞ (co F(y)) ⊆ F co ∞ (y); the last part of statement (iii) follows from (5.13)- (5.14) and from the fact that if x(•) ∈ Ω ∞ , then it is Lipschitzian, hence ∅ ≠ K ± x(0; 1) ⊂ K ± F (y) ⊆ co F(y), K ± x(0; 0 + ) = {(0, 0)}.Since statement (iv) follows obviously from the relations (5.49) in Remark 5.3 and from the statements (i)-(iii), the theorem is completely proved.Remark 6.2.Due to the relations in (2.7), the inequalities in (6.1) imply also the following ones, expressed in the terms of the "intermediate" quasitangent extreme derivatives in (2.4), (2.1): which may be proved directly for the orientor fields F(•) that satisfy Hypothesis 3.2, using the corresponding relations in Theorems 3.4, 5.1.We note that according to statement (iv) in Theorem 5.1, the inequality in (6.2) may be extended to the points y ∈ Y i 0 in the case where F(•) has the affine growth property in (3.8).
We recall that certain variants of the contingent inequalities in (6.8)-(6.14)have been obtained in [4,10,11,13], and so forth, under much more restrictive hypotheses on the data while the stronger variants in (6.1)-(6.7)have been obtained for the first time in [14] for nonautonomous Mayer type, continuously parameterized optimal control problems.
As already stated, the differential inequalities in Theorem 6.1, as well as the more general ones in Theorem 5.1, may be expressed in weaker forms in terms of the corresponding semidifferentials in (2.6) which, in view of the relations in (2.2), (2.5), may equivalently be defined by in terms of the left variants of the directional derivatives and tangent cones.
In this setting, it is convenient to use the associated Hamiltonians p, v +v 0 , (6.18) noting that the horizon Hamiltonian, H ∞ (x, p), may be defined only at the points x ∈ Y at which F(x) ⊂ R n × R is unbounded (otherwise D ∞ (co F(x)) = {(0, 0)}); we note also the fact that H(x, p) := inf v=(v,v 0 )∈co F(x) [ p, v +v 0 ].
Corollary 6.3.The semidifferentials in (2.6), (6.17)   Remark 6.4.The fact that the results in Corollary 6.3 are weaker forms of the differential inequalities in Theorem 6.1 is due not only to the fact that they are direct consequences of Theorem 6.1 but also to the fact that the corresponding semidifferentials in (2.6), (6.17) may have empty values at some points at which the differential inequalities in Corollary 6.3 are trivially satisfied while the corresponding ones in Theorem 6.1 may still provide some useful information.
On the other hand, we point out the fact that the relations in (6.21), (6.22) show that the proper value function W 0 (•) in (4.6) is not only a viscosity solution of the equivalent generalized HJB equation but also a strict viscosity subsolution on Y 0 since it satisfies (6.25) and, moreover, a strict viscosity solution on the subset Y − 0 in (5.3) since it satisfies the relation in (6.26).These additional properties may explain some of the more recent results in the very abundant theory of viscosity solutions (see, e.g., Bardi and Capuzzo-Dolcetta [3], Plaskacz and Quincampoix [21], etc.), in which, to the basic axioms in the original definitions of viscosity solutions (see, e.g., Crandall and Lions [8], etc.), one adds different types of other properties to obtain existence and uniqueness; however, simple examples (see, e.g., Miricȃ [14], Example 7.1 below, etc.) show that the hypotheses insuring uniqueness of the value functions as viscosity solutions are rather restrictive.
Moreover, as shown in [14], the very special properties of the Clarke extreme directional derivatives in (2.4), (2.1) of a real locally Lipschitz function g(•) at an interior point, x ∈ Int(X), of its domain, show that the results in Theorem 6.1 imply, in particular, the fact that under the suitable but rather restrictive hypotheses in [7], under which W 0 (•) is locally Lipschitz at y ∈ Int( Y 0 ), it satisfies the relation inf

An illustrative example.
Although some of the aspects in the previous sections are illustrated by [14,Examples 8 and 11], we consider in some detail the following particular problem which is actually an autonomous variant of [5, problem 1.4.5]; the aim of this example is to illustrate, in particular, the rather irregular structure of the effective domain in (5.1)-(5.3) of the proper value function in (4.6) and also the essential nature of some of the differential inequalities in Theorems 5.1, 6.1.
To prove (7.7) we consider y = (y 1 ,y 2 ) ∈ (−∞, 0) × R, m ≥ 1 and note that from (7.6) it follows that for the initial point y m := (0,y 2 − my 1 ) there exists an admissible control u m (•) ∈ ᐁ α (y m ) such that Ꮿ(y m ; u m (•)) < 1; next, we consider the following control functions: whose corresponding trajectories .11)are admissible in the sense of (7.2); moreover, the values of the cost functional in (7.1) are given by and (7.7) is proved.For the complete characterization of the value function it remains to prove the optimality of the extremals in (7.4) and therefore of the proper value function, W 0 (•), in (7.5); to this end one may try first to use a rather involved argument using corresponding results concerning the existence of optimal controls and the proof of the uniqueness of the extremals, that satisfy the PMP (see, e.g., Cesari [5]) for each initial point y ∈ Y 0 ; at first sight, it is difficult to see why the same arguments are not valid for the neighboring initial points y ∈ Y 0 \ Y 0 in (7.6), (7.7) which, clearly, do not have optimal controls.
An apparently easier argument for the optimality of the extremals in (7.4) may be obtained in the framework of dynamic programming using, for instance, the "verification" Theorem 5.6 in [12], for lower semicontinuous value functions since the extended The above-mentioned verification theorem may be applied to prove the optimality of the extremals in (7.4) in the restricted class of Lipschitzian trajectories, Ω α ⊆ Ω ∞ (generated by the class ᐁ ∞ of bounded, measurable control functions), since, as one may easily verify, the subset Y 0 ⊂ Y 0 in (7.5), (7.13) is invariant with respect to the control system in (7.2) and the lower contingent derivatives of the value function in (7.13) at the endpoints, y 0 = (0, 0) ∈ Y 0 \ Y − 0 , ξ = (1, 0) ∈ Y 1 , are given, respectively, by Next, using suitable ad hoc arguments one may prove that the associated convexified upper limits in (3.7) of the extended orientor field, F(•), in (5.5), which in our case is defined by F(x) = (1, u), x 1 u 2 ; u ∈ R , x∈ Y := (−∞, 1) × R ∪ (1, 0) , (7.16) are given by the formulas one may not apply either of Theorems 5.5, 5.6 in [12] to obtain the optimality in a larger class, ᐁ α ⊃ ᐁ ∞ , of admissible controls; however, in this case one may try to use a direct argument and the particular form in (7.5) of the value function to prove the fact that the real functions ω x (•) in (4.7) are increasing in the largest class, ᐁ 2 = L 2 , of square integrable control functions.show that at the (rather singular) points y = (0,y 2 ) ∈ Y 0 the sets of contingent directions in (3.4) are given by K + F (0, 0) = co F(0, 0) = {1}×R ×{0} ⊂ F co (0, 0) = {1}×R × R, K ∞ F (0, 0) = D ∞ co F(0, 0) = {0}×R ×{0} ⊂ F co ∞ (0, 0), (7.22) which prove that the inclusions in (3.7) may be strict if F(•) is not upper hemicontinuous at y ∈ Y 0 .
In particular, Example 7.1 shows that Theorem 5.1(iv) may still hold even if F(•) does not have the affine growth property in (3.8).
Finally, one may note that from (6.18), (7.22) it follows that the horizon Hamiltonian in (6.18) is given in this case by the formula (7.23) which shows that the generalized HJB equation in (6.27) may be equivalent to the usual one in (6.28), at least on the subset Y − 0 in (5.3) of the relative interior of the effective domain, Y d 0 , of the value function.
One may note also that the function W 1 (x) ≡ 0 is another smooth classical solution (hence also a viscosity solution) of the HJB equation in (6.28).
Other properties illustrating the essential nature of some of the results in Theorem 5.1 may be obtained from the formulas in (7.15)- (7.22).

Hypothesis 3 . 1 .Hypothesis 3 . 2 .Hypothesis 3 . 3 .
There exists r > 0 such that B r (y) ⊂ Y 0 and the restriction F(•)|B r (y) has closed values and is Hausdorff continuous; here B r (y) := {x ∈ R n ; x − y < r } denotes the ball of radius r > 0 centered at y.There exists r > 0 such that B r (y) ⊂ Y 0 and the restriction F(•)|B r (y) has closed values and is Hausdorff locally Lipschitz.There exists r > 0

Theorem 5 . 1 .
The proper value function W 0 (•) and the extended value function W (•) in (4.6) satisfy the following differential inequalities.