Discrete Dynamics in Nature and Society

Volume 2013 (2013), Article ID 769368, 12 pages

http://dx.doi.org/10.1155/2013/769368

Research Article

Weak KAM Solutions of a Discrete-Time Hamilton-Jacobi Equation in a Minimax Framework

Porfirio Toledo

Facultad de Matemáticas, UV Zona Universitaria, 91090 Xalapa, Ver, Mexico

Received 28 January 2013; Accepted 31 May 2013

Academic Editor: Stepan Agop Tersian

Copyright © 2013 Porfirio Toledo. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract.

The purpose of this paper is to study the existence of solutions of a Hamilton-Jacobi equation in a minimax discrete-time case and to show different characterizations for a real number called the critical value, which plays a central role in this work. We study the behavior of solutions of this problem using tools of game theory to obtain a “fixed point” of the Lax operator associated, considering some facts of weak KAM theory to interpret these solutions as discrete viscosity solutions. These solutions represent the optimal payoff of a zero-sum game of two players, with increasingly long time payoffs. The developed techniques allow us to study the behavior of an infinite time game without using discount factors or average actions.

1. Introduction

The Hamilton-Jacobi equation is a very important tool for the study of Lagrangian systems and control theory. In the case of convex Lagrangians, Fathi, Mather, and Mañe have perfected several techniques to understand the solutions of this equation ([1–7]). These techniques can be translated to a discrete case where, instead of a Lagrangian, there is a function of two variables in the configuration space.

The aim of this paper is to study a discrete-time system that emulates a convex and concave Lagrangian. We need to study the behavior of solutions of a discrete-time version of a Hamilton-Jacobi equation in the context of game theory. A real number, called the critical value, plays a central role in this work; this number is the asymptotic average action of optimal trajectories. We study the existence and characterization of solutions of a Hamilton-Jacobi equation for this kind of game. A relevant aspect is that these techniques allow us to study infinite-time games without using discount factors or average actions.

Consider the compact metric spaces and , which are the state spaces for players and , respectively. For , a continuous function , and a Lipschitz function , we define the static game payoff by

The function is the amount that player obtains from and vice versa; so player wants to maximize the payoff , while player wants to minimize it. The lower game value is defined by

It is observed in [8] that it is possible to express the lower game value as follows:

where the function is called a strategy for player . These functions are very important in this work and will be defined in general for a zero-sum game with finite horizon.

For , consider the finite horizon problem, where players and alternate their moves beginning at initial state and setting with them the state trajectories determined by finite sequences and , denoted by and , respectively. The state of the system depends on the initial condition and the players decisions to get from to , where . For , we will denote by

the sets of finite sequences in and , with initial states and . We define the action by

and the game payoff by

Then, the lower game value with finite horizon and initial state is

However, can be defined directly using progressive strategies (see [8–10]). The idea is to use a tool that allows player to select one point knowing the current and past opponent choices, without knowing the future choices. To this end, we define a progressive strategy for player as a function which satisfies the following property: for each such that and , we have

where is the th element of sequence . The set of progressive strategies for player is denoted by

Finally, the lower game value can be expressed as

where and . This function is called Lax operator and will be essential for understanding the asymptotic behavior of game payoffs. The Proposition 6 shows that

In the next section of this paper, the objective is to prove the existence of a kind of “fixed point” of the Lax operator. We will adapt the methods developed in the work of Fathi ([3]) about weak KAM theory, now in a minimax context of game theory. It will be necessary to prove that the Lax operator , defined as in (10), satisfies some important properties: the semigroup property that is also call Dynamic Programming Principle (see [8–10]), a regularity property, monotony, and weak contraction.

Theorem 1. If and are compact metric spaces and is a Lipschitz function, then there exist a Lipschitz function and a unique constant , such that

The functions in the previous result are called fixed points of the Lax operator with critical value . These fixed points are solutions of the following discrete-time Hamilton-Jacobi equation associated to :

See [11] for a minimal discrete-time case.

In the third section, we will consider an infinite horizon problem, which will help us to find a different characterization of critical value . To this end, we define the lower Peierls barrier as follows, given a real number and an initial state :

where and , see [3, 12, 13].

It is possible to prove that there exists a number , such that

where the value of does not depend on . The critical value has a new interpretation characterized by the following result.

Theorem 2. If and are compact metric spaces, is a Lipschitz function, and , then and , where is the critical value.

Analogously, we can define the upper Peierls barrier by

for which we can obtain similar results.

Considering that and , for every , we prove that

and therefore,

In addition, using a solution of the Hamilton-Jacobi equation, we can find an optimal strategy, from which optimal sequences are obtained for both players for all times. In particular, we prove the following result.

Theorem 3. If and are compact metric spaces, is a Lipschitz function, and , then there are sequences and such that

where and , where is the critical value.

From this, it follows immediately that

The sequences obtained in the previous result satisfy the Bellman principle of optimality, because from each state in the optimal trajectory, the complement remains optimal. The optimality of the remaining decisions from depends on this last state, which is the result of the previous decisions.

In last section, we will show another characterization for the solutions of the Hamilton-Jacobi equation.

For , a function is called -supersolution if

Analogously, a function is called -subsolution if

The function is a -solution if is -supersolution and -subsolution (see [4]).

Clearly, is a -solution if and only if is a fixed point of the Lax operator with critical value . According to Theorem 1, if there exists a unique constant for fixed points of the Lax operator, then there is only one kind of -solutions, in fact for . The -solutions are called critical solutions. From this, we get another characterization of , which appears in the following result.

Theorem 4. If and are compact metric spaces, is a Lipschitz function, and is a fixed point of the Lax operator with critical value , then

where

Finally, we will show that is a critical solution and therefore a solution of the Hamilton-Jacobi equation.

2. Lax Operator

Let and be compact metric spaces, a Lipschitz function, and . Consider a zero-sum game of two players, and , with finite horizon , where and represent the state spaces and sequences and determine the state trajectories, respectively, for players and . The function , defined by

is the game action. For and , the game payoff is defined by

If is the set of progressive strategies for player , the Lax operator is the lower game value, defined by

with and . We prove the following regularity property for the Lax operator.

Lemma 5. The Lax operator is -Lipschitz, for any and , where is the Lipschitz constant of .

Proof. We will consider , , and . There exists such that

where .

For a finite sequence , we define as

For this strategy, we have

where . It follows that there is such that

We define as . Combining (28) with (31), we obtain

Since and , for , we have

where is the Lipschitz constant of and is a distance in . Similarly, we obtain the opposite inequality. Thus,

From above result, we prove the semigroup property for the Lax operator, also called the Principle of Dynamic Programming.

Proposition 6 (Semigroup Property). For ,

where and .

Proof. We will consider and . To prove the relation , we will see that, for every , the inequalities and take place simultaneously.

(I) Let us first prove that . To do this, take such that

where .

On the other hand, for each ,

where and . Hence, there is such that

We will define a using the previous strategy and the family . Given a sequence in , the associated sequence in is defined as follows: the fist terms are given by the strategy and the second part is given by , for a specific . More precisely, for ,

where , , , and . Taking any and combining (36) with (38), we obtain

Therefore,

with . This implies that

and consequently .

(II) It remains to prove that . To this end, we will consider such that

where .

We will define as the -time restriction of . Given a sequence , the associated sequence in is defined as follows: the terms are given by the fist terms of strategy applied to some extension of in , adding any sequence . Since is a strategy, it follows that the function is well defined and it is a strategy. More precisely, for , taking any , we define as

for , where . So

where . Consequently, there is such that

Once that is fixed so will be . We now define , for , as

for , where . We have

where . Therefore, there is such that

Define as

Combining (43) and (46) with (49) we obtain:

for this reason .

Lemma 7. If , then

(1)(monotony) if , then ;(2)if , then ;(3)(weak contraction) .

Proof. The first two properties are a direct consequence of the definition. For last the claim, choose , then

The previous items imply that

So , for every . As a result,

Theorem 1 is analogue to Weak KAM Theorem. We use the same argument of Fathi (see [3]) to prove the existence of fixed points of this discrete-time case of the Lax operator.

According to this result, we say that is a fixed point of the Lax operator with critical value .

Proof. (I) To prove the existence of a fixed point, we will use the same argument used by Fathi ([3]) for the continuous-time case.

Let be the quotient of the set of continuous functions modulo constants, . The norm is defined as

for . Then, is a Banach space. We define the operator as

which satisfies the weak contraction property. Thus, , with , is a contraction; so there are fixed points of . As is equi-Lipschitz, since Arzela-Ascoli theorem, it follows that has a uniformly convergent subsequence when . Then, has a fixed point , which in terms of means that there is a constant such that

for each positive integer . From the semigroup property, there is such that

for all positive integers .

Function is a Lipschitz function, because is also a Lipschitz function. In fact, according to Lemma 5, is -Lipschitz, where is the Lipschitz constant of .

(II) It remains to prove that the number is unique. Let be positive. Suppose there are and , , such that , with . Choose such that

For this strategy, we also have that

So there is , such that

Therefore, using the hypothesis, we obtain

Considering the subtraction of the previous inequalities, we obtain following relations:

then

On the other hand, because , we have when , which is a contradiction.

Considering the one-step case (see [8]), we can rewrite the Lax operator as

If is a fixed point with critical value , then is a solution for the following discrete-time analog of the Hamilton-Jacobi equation:

Conversely, if is a solution of the previous Hamilton-Jacobi equation, then is a fixed point of the Lax operator with critical value . According to semigroup property and relation in Lemma 7, it follows that is a fixed point of with critical value , for all .

3. Peierls Barrier

Let us now study an infinite horizon problem for players and . Let and be compact metric spaces, and let be a Lipschitz function. For , let us consider the function defined as

where and .

For , the lower Peierls barrier is

Proposition 8. For every , one has the following:

(1)if for some , then , for all ;(2)if for some , then , for all .

Proof. Let be positive and consider .

(1)Let us take and suppose that . Hence, there is such that

 Therefore, there is such that if , then

 Let be a strategy. We define as follows: for ,

 Therefore, . Hence, there is such that

 If we define , then

 in which it follows that , for every , where . Then,

 so .(2) Let us take and suppose that . There is such that if , then

 Therefore, there is such that

 Taking , define as

 So, considering ,

 Since the previous relation is valid for every , it follows that

 Hence,

In order to see that has a radical change on its values, for a particular value of , we prove the following statement.

Lemma 9. There exists large enough, such that

(1). (2).

Proof. Considering , we have . If and , then , for each . Hence,

The other relation can be obtained similarly.

It is clear that is monotone in .

Lemma 10. Let , and be real numbers such that , then

(1)if , then ;(2)if , then ;(3)moreover, if , then

Corollary 11. There exists , such that

Proof. From the previous results, it follows that there are

Given , choose such that and , then . Therefore, , so

Suppose that , then there are and such that and

Lemma 10 shows that and ; this is a contradiction. Hence,

Theorem 2 provides us a different characterization for the critical value .

Proof. Let be a fixed point of the Lax operator with critical value . For and ,

where and . Therefore,

and then, for every ,

So ; on that account, . Analogously, from (88), we obtain

consequently, for each ,

Hence, and in addition .

We can introduce the upper Peierls barrier defined by

By similar arguments to the ones we made for , we obtain analogous properties for and the following result.

Theorem 12. If and are compact metric spaces, is a Lipschitz function, and , then

and , where is the critical value.

On account of lower and upper Peierls barrier properties, we have if .

Corollary 13. If and are compact metric spaces, is a Lipschitz function, and , then

where is the critical value.

Proof. There is such that

for . Let be the maximum of numbers , , . According to Theorems 2 and 1², we have , so , and in addition,

for all .

Corollary 14. If and are compact metric spaces, is a Lipschitz function, and , then

where is the critical value.

Theorem 3 states that can be written in terms of actions for certain special sequences, using a solution of the Hamilton-Jacobi equation.

Proof. Consider , , and solutions of the Hamilton-Jacobi equation (66). Because and are compact spaces, for each , given , we can choose

where . We may now choose

On account of is the solution of (66), we have

Consequently, for each , we have

where and .

By the compactness of and and because we obtain that

from which we conclude that

for all .

We can obtain another characterization of the critical value in terms of optimal sequences in the previous theorem.

Corollary 15. If and are compact metric spaces, is a Lipschitz function, and , then there are sequences and such that

where and , where is the critical value.

4. Critical Solutions

Let and be compact metric spaces, and let be a Lipschitz function.

Definition 16. For , a function is called -supersolution if

Analogously, a function is called -subsolution if

The function is a -solution, if is -supersolution and -subsolution.

It is evident that is a -solution if and only if is a fixed point of the Lax operator with critical value ; in this case according to Theorem 1. The -solutions are called critical solutions.

Proposition 17. If and are compact metric spaces, is a Lipschitz function, , and , then

(1) is -supersolution if and only if there exists such that

 for all ;(2) is -subsolution if and only if for every , there exists such that

Proof. Consider ,

(1) satisfies (108) if and only if there is such that

 so

 where and ;(2) satisfies (109) if and only if for every , we have

 Next,

 where and .

We have another consequence of the previous result: if , the critical solutions satisfy the following statements:

(1)for every strategy , there exists such that

(2)there exists a strategy such that

We have some properties of -super and -subsolutions.

Lemma 18. For and ,

(1)if is -supersolution and , then is -supersolution;(2)if is -subsolution and , then is -subsolution;(3)if the sets are defined as

then .

Proof. The first and second items are clear. To prove the third claim, choose and . Then, there is such that

for all . Because , there is such that

Therefore, , and then .

By definition, if is a fixed point of the Lax operator with critical value , then is a critical solution. Therefore, , and according to Lemma 18, we have a new characterization of the critical value (see Theorem 4).

We are interested in showing that the lower Peierls barrier is a critical solution. To this end, we consider the behavior of extremal sequences for lower Peierls barrier defined as follows.

Definition 19. We define a couple extremal for , if and satisfies the following conditions:

(1)for every strategy , there exists such that

(2)for every ,

The following result tells us that the lower Peierls barrier is a critical solution and therefore a solution of the Hamilton-Jacobi equation.

Theorem 20. is a critical solution.

Proof. Consider . According to Theorem 2, .

(I) We begin by proving is -supersolution. Let us take and let be extremal for , then

where .

For every , define as

then

There is such that

Choose a subsequence such that and

extracting a subsequence if necessary, we can assume that

where denotes the set of infinite sequences in , with initial condition . Therefore,

where . Because is continuous, then

which proves that there is such that

for all . Hence, is -supersolution.

(II) We proceed to show that is -subsolution. Let us take . For each , let be extremal for , then

Observe that depends only on , because . Suppose that .

For , we can define a strategy as follows:

where . So

We can choose extremal for

and a subsequence such that

If , then

According to the above relations and the continuity of , if , there is such that

Therefore, is a -subsolution.

References

A. Fathi, “Théorème KAM faible et théorie de Mather sur les systèmes lagrangiens,” Comptes Rendus de l'Académie des Sciences, vol. 324, no. 9, pp. 1043–1046, 1997.

A. Fathi, “Sur la convergence du semi-groupe de Lax-Oleinik,” Comptes Rendus de l'Académie des Sciences, vol. 327, no. 3, pp. 267–270, 1998.

A. Fathi, Weak KAM Theorem in Lagrangian Dynamics, Preliminary Version, 2008.

A. Fathi and A. Siconolfi, “Existence of critical subsolutions of the Hamilton-Jacobi equation,” Inventiones Mathematicae, vol. 155, no. 2, pp. 363–388, 2004.

R. Mañé, “Global variational methods in conservative dynamics,” in 18 Colóquio Brasileiro de Matemática, Instituto de Matemática Pura e Aplicada (IMPA), Rio de Janeiro, Brazil, 1993.

R. Mañé, “Lagrangian flows: the dynamics of globally minimizing orbits,” Boletim da Sociedade Brasileira de Matemática, vol. 28, no. 2, pp. 141–153, 1997.

J. N. Mather, “Action minimizing invariant measures for positive definite Lagrangian systems,” Mathematische Zeitschrift, vol. 207, no. 2, pp. 169–207, 1991.

W. H. Fleming and H. M. Soner, Controlled Markov Processes and Viscosity Solutions, vol. 25, Springer, New York, NY, USA, 2nd edition, 2006.

M. Bardi and I. Capuzzo-Dolcetta, Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman Equations, Birkhäuser, Boston, Mass, USA, 1997.

L. C. Evans and P. E. Souganidis, “Differential games and representation formulas for solutions of Hamilton-Jacobi-Isaacs equations,” Indiana University Mathematics Journal, vol. 33, no. 5, pp. 773–797, 1984.

D. A. Gomes, “Viscosity solution methods and the discrete Aubry-Mather problem,” Discrete and Continuous Dynamical Systems, vol. 13, no. 1, pp. 103–116, 2005.

G. Contreras and R. Iturriaga, Global Minimizers of Autonomous Lagrangians, Instituto de Matemática Pura e Aplicada (IMPA), Rio de Janeiro, Brazil, 1999.

G. Contreras, “Action potential and weak KAM solutions,” Calculus of Variations and Partial Differential Equations, vol. 13, no. 4, pp. 427–458, 2001.