As an important tool in theoretical economics, Bellman equation is very powerful in solving optimization problems of discrete time and is frequently used in monetary theory. Because there is not a general method to solve this problem in monetary theory, it is hard to grasp the setting and solution of Bellman equation and easy to reach wrong conclusions. In this paper, we discuss the rules and problems that should be paid attention to when incorporating money into general equilibrium models. A general setting and solution of Bellman equation in monetary theory are provided. The proposed method is clear, is easy to grasp, is generalized, and always leads to the correct results.

In recent years, many economists applied business cycle approaches to macroeconomic modeling so that monetary factors could be modeled into dynamic general equilibrium models. As an important method of monetary economics modeling, infinite horizon representative-agent models provide a close link between theory and practice; its research framework can guide practice behavior and be tested by actual data. This method can link monetary economics and other popular models for studying business cycle phenomena closely. There are three basic monetary economics approaches introducing money to economic general equilibrium models in the infinite horizon representative-agent framework. First, it is assumed that utility could be yielded by money directly so that the money variable has been incorporated into utility function of the representative-agent models [

Dynamic optimization is the main involved issue during the modeling process. It is represented and solved by Bellman equation method, namely, the value function method. The method will obtain a forward-looking household’s path to maximize lifetime utility through the optimal behavior and further relevant conclusions. The setting of Bellman equation is the first and crucial step to solve dynamic programming problems. It is hard to grasp the setting and solution of Bellman equation and easy to reach wrong conclusions since there is not a general method to set Bellman equation or the settings of Bellman equation are excessively flexible. Walsh [

In this paper, we provide a set of general setting and solution methods for Bellman equation with multipliers. It is very clear and easy to grasp. The most important thing is that the proposed method can always lead to correct results. We apply our method to monetary general equilibrium models that are in the framework of the first two basic monetary economic approaches which incorporate money into economic general equilibrium model and make some extensions. At the same time, we compare our method with other current methods of setting and solution for Bellman equation to display the clarity and correctness of the proposed method. It is assumed that all the relevant assumptions of applying Bellman equation are satisfied. This is to ensure the feasibility of analysis and solution. For the convenience of discussion and suitable length of the paper, we mainly discuss the certainty linear programming problems. The Bellman equation’s setting and solution of uncertainty problems are similar to those with certainty problems essentially.

First of all, we provide the theoretical details of the new general method and steps when applying Bellman equation to solve problems. It has many advantages. The relationship of the results is very clear through the connection of multipliers. There are no tedious expansions during the derivations. It differs from the expansion method because it does not need to consider which control variable should be replaced. The technical details about the equivalence of Bellman equations and dynamic programming problems and the solvability of set problems can be found in [

List the expression of target problem.

Consider the dynamic programming problem as

Set up Bellman equation with multipliers to express dynamic optimization problem in Step

Compute the partial derivatives of all control variables on the right side of the equation at Step

During the derivation, it should be taken into account that next period state variables can be represented by other control variables according to the constraints, that is, to expand

By the envelope theorem, take the partial derivatives of control variables at time

Still, it does not need to expand

Obtain the new relevant results about target problem through recursion and substitution according to the above results.

We could combine several state variables to one as you need according to specific economic significance and constraints when there is no need to describe the relevant economic significance of state variables one by one. For example,

The steps above give a general setting and solution of Bellman equation. These can be summarized as follows: first, set Bellman equation with multipliers of target dynamic optimization problem under the requirement of no overlaps of state variables; second, extend the late period state variables in

Different from some current settings which allow overlap of state variables in value function, our method does not permit overlaps. In fact, overlap of state variable is easy to reach wrong conclusions or it has to find some particular skills to get the right results [

Now we will take several discrete time dynamic optimization problems that are under framework of the basic two methods of incorporating money into general equilibrium models as examples to show the applications of our setting and solution method and compare with current popular methods.

The basic Money-In-Utility model has few features. The labor-leisure options of families are ignored temporarily in utility function. Only family consumption and real money balances are involved in utility function; that is, real money balances yield utility directly. We ignore the uncertainty of currency impact and technological changes temporarily for convenience.

The total present utility value of family life cycle is

This problem can be solved by the expansion method below according to Walsh [

Using (

Replace

Compute the partial derivatives of control variables to derive first-order conditions:

Another expansion method will replace

Compute the partial derivatives of control variables to derive first-order conditions as

Using the envelope theorem and computing the derivative with respect to state variable

Now, we use our proposed steps of setting and solution of Bellman equation to solve the above basic Money-In-Utility problem. First, let the Bellman equation with multiplier

Second, computing the partial derivatives for the control variables, we obtain the first-order conditions as

Third, using the envelope theorem and computing the derivatives of both sides of Bellman equation with respect to state variable

Finally, based on the above results, it is easy to get

This expression indicates that the marginal benefit of increased money holdings should be equal to the marginal utility of consumption on period

These results with multiplier are open-and-shut, and it is easy to find the economic significance of marginal utility of consumption. Comparing with current approach mentioned above, during the process of solving problem, there is no need to consider which variable should be replaced. There are no tedious expanded expressions. Using expansion method, if we replace

In Shopping-Time Model, shopping time is a function of consumption and money balances. Because consumption needs shopping time, leisure is reduced. Household utility is assumed to depend on consumption and leisure. Consumption can not only yield utility directly but also decrease utility indirectly. In this section,

The household’s intertemporal objective is maximum discounted utility subject to resource constraint as

Because controllable factors, labor supply, and consumption affect labor supply, output is affected not only by state variable

According to Walsh [

Computing the partial derivatives with respect to control variables, we get

Computing the partial derivatives of both sides of the value equation with respect to the state variables

Now, we use our proposed method to solve the above Shopping-Time Model problem. First, let the Bellman equation with multiplier

Using the envelope theorem and computing the derivatives with respect to the state variables

Now, we use the above results to compute the opportunity cost of holding money. Since the utility function is

Obviously, the expressions of the derived first-order conditions by previous method seem to be tedious and messy, and it is not so easy to compute the relevant results such as the opportunity cost of holding money. Our proposed method is comparatively neat and can easily obtain relevant results correctly.

In basic Cash-In-Advance model, money is used to purchase goods. Money cannot yield utility itself, but the consumption of future can yield utility. Svensson [

The budget constraint is rewritten in real terms as

In monetary theory, constraints are expressed as inequality frequently. This constraint describes that the representative agent’s time

Output is only affected by state variable

Let

The solution of this problem is similar to the application of the proposed method in Sections

Note that Walsh [

Assuming that money is used to purchase consumption goods and investments, the Cash-In-Advance constraint becomes

Output is affected not only by state variable

The first-order conditions are

Using the envelope theorem and computing the derivatives with respect to state variables, we get

When the question is to derive the effect of inflation rate on the steady-state capital-labor ratio, that is, the steady-state relationship of

Then

From (

Rewriting the aggregate production as

Using (

Rewriting this equation, we get

In steady-state,

This steady-state capital-labor ratio is derived by using current prevailing methods. However, it is a wrong result. We will provide the correct result by using the proposed method of this paper.

Output is affected by state variable

First, set up value function with multipliers:

Second, compute the derivatives of state variables and derive first-order conditions:

Third, compute the partial derivatives with respect to

Finally, derive the steady-state capital-labor ratio by the results above:

The process of deriving this ratio is similar to the deriving process of (

As an important tool in theory economics, Bellman equation is very powerful in solving optimization problems of discrete time and is frequently used in monetary theory. It is hard to grasp the setting and solution of Bellman equation and easy to reach wrong conclusions since there is no general method to set Bellman equation or the settings of Bellman equation are excessively flexible. In this paper, we provide a set of general setting and solution methods for Bellman equation with multipliers. In the processes of solving monetary problems, comparing with other current methods in classic reference, our proposed method demonstrates its features of clarity, validity, correct results, easy operation, and generalization. Bellman equation is used not only in monetary problems but also in almost every dynamic programming problem associated with discrete time optimization. Our future work is to study the applicability of the proposed method in this paper in other areas.

The authors declare that there is no conflict of interests regarding the publication of this paper.

Wanbo Lu’s research is sponsored by the National Science Foundation of China (71101118) and the Program for New Century Excellent Talents in University (NCET-13-0961) in China.