We consider a discrete-time Markov chain with state space {1,1+Δx,…,1+kΔx=N}. We compute explicitly the probability pj that the chain, starting from 1+jΔx, will hit N before 1, as well as the expected number dj of transitions needed to end the game. In the limit when Δx and the time Δt between the transitions decrease to zero appropriately, the Markov chain tends to a geometric Brownian motion. We show that pj and djΔt tend to the corresponding quantities for the geometric Brownian motion.
1. Introduction
Let {X(t),t≥0} be a one-dimensional geometric Brownian motion defined by the stochastic differential equation
dX(t)=μX(t)dt+σX(t)dB(t),
where μ∈ℝ, σ>0, and {B(t),t≥0} is a standard Brownian motion. Assume that X(0)=x∈(1,N), where N∈ℕ (for simplicity), and defineτ(x)=inf{t>0:X(t)=1orN∣X(0)=x}.
As is well known (see, e.g., Lefebvre [1, page 220]), the probability
p(x):=P[X[τ(x)]=N]
satisfies the ordinary differential equation
12σ2x2p′′(x)+μxp′(x)=0,
subject to the boundary conditions
p(1)=0,p(N)=1.
We easily find that, if c:=μ/σ2≠1/2,
p(x)=x1-2c-1N1-2c-1for1≤x≤N.
When c=1/2, the solution is
p(x)=lnxlnNfor1≤x≤N.
Moreover, the function
m(x):=E[τ(x)]
satisfies the ordinary differential equation (see, again, Lefebvre [1, page 220])
12σ2x2m′′(x)+μxm′(x)=-1,
subject to
m(1)=m(N)=0.
This time, if c≠1/2 we find that
m(x)=2(1-2c)σ2{lnx-lnNx1-2c-1N1-2c-1}for1≤x≤N
and, for c=1/2,
m(x)=lnxσ2(lnN-lnx)for1≤x≤N.
Now, it can be shown (see Cox and Miller [2, page 213]) that the discrete-time Markov chain {XmΔt,m=0,1,…} with state space {1,1+Δx,…,1+kΔx}, where k is such that 1+kΔx=N, and transition probabilities
p1+jΔx,1+(j+1)Δx=12A{(1+jΔx)2σ2+(1+jΔx)μΔx},p1+jΔx,1+(j-1)Δx=12A{(1+jΔx)2σ2-(1+jΔx)μΔx},p1+jΔx,1+jΔx=1-1A(1+jΔx)2σ2,
where j∈{1,…,k-1}, converges to the geometric Brownian motion {X(t),t≥0} as Δx and Δt decrease to zero, provided that(Δx)2=AΔt,(jΔx)2<A∀j∈{0,…,k}.
Remarks 1.1.
(i) We assume that all the probabilities defined by (1.13) are well defined; that is, they all belong to the interval [0,1].
(ii) The condition in (1.15) implies that (Δx)2<A/k2.
Let
Tj:=inf{m>0:XmΔt=1orN∣X0=1+jΔx},pj:=P[XTj=N].
In the next section, we will compute the quantity pj for j∈{1,…,k-1}. We will show that pj converges to the function p(x) for the geometric Brownian motion as Δx decreases to zero and k tends to infinity in such a way that 1+kΔx remains equal to N.
In Section 3, we will compute the mean number of transitions needed to end the game, namely,
dj:=E[Tj].
By making a change of variable to transform the diffusion process {X(t),t≥0} into a geometric Brownian motion with infinitesimal mean equal to zero and by considering the corresponding discrete-time Markov chain, we will obtain an explicit and exact expression for dj that, when multiplied by Δt, tends to m(x) if the time Δt between the transitions is chosen suitably.
The motivation for our work is the following. Lefebvre [3] computed the probability p(x) and the expected duration m(x) for asymmetric Wiener processes in the interval (-d,d), that is, for Wiener processes for which the infinitesimal means μ+ and μ-, and infinitesimal variances σ+2 and σ-2, are not necessarily the same when x>0 or x<0. To confirm his results, he considered a random walk that converges to the Wiener process. Lefebvre's results were extended by Abundo [4] to general one-dimensional diffusion processes. However, Abundo did not obtain the quantities pj and dj for the corresponding discrete-time Markov chains. Also, it is worth mentioning that asymmetric diffusion processes need not be defined in an interval that includes the origin. A process defined in the interval (a,b) can be asymmetric with respect to any a<c<b.
Next, Lefebvre and Guilbault [5] and Guilbault and Lefebvre [6] computed pj and dj, respectively, for a discrete-time Markov chain that tends to the Ornstein-Uhlenbeck process. The authors also computed the quantity pj in the case when the Markov chain is asymmetric (as in Lefebvre [3]).
Asymmetric processes can be used in financial mathematics to model the price of a stock when, in particular, the infinitesimal variance (i.e., the volatility) tends to increase with the price of the stock. Indeed, it seems logical that the volatility is larger when the stock price X(t) is very large than when it is close to zero. The prices of commodities, such as gold and oil, are also more volatile when they reach a certain level.
In order to check the validity of the expressions obtained by Abundo [4] for p(x) and m(x), it is important to obtain the corresponding quantities for the discrete-time Markov chains and then proceed by taking the limit as Δx and Δt decrease to zero appropriately. Moreover, the formulas that will be derived in the present paper are interesting in themselves, since in reality stock or commodity prices do not vary completely continuously.
First passage problems for Markov chains have many applications. For example, in neural networks, an important quantity is the interspike time, that is, the time between spikes of a firing neuron (which means that the neuron sends a signal to other neurons). Discrete-time Markov chains have been used as models in this context, and the interspike time is the number of steps it takes the chain to reach the threshold at which firing occurs.
2. Computation of the Probability pj
Assume first that Δx=1, so that the state space is {1,2,…,N} and the transition probabilities become pj,j+1=12A{j2σ2+jμ},pj,j-1=12A{j2σ2-jμ},pj,j=1-j2σ2A
for j∈{2,…,N-1}. The probability defined in (1.17) satisfies the following difference equation:
pj=pj,j+1pj+1+pj,j-1pj-1+pj,jpj.
That is,
2jpj=(j+c)pj+1+(j-c)pj-1,
where c=μ/σ2. The boundary conditions are
p1=0,pN=1.
In the special case when μ=0, (2.3) reduces to the second-order difference equation with constant coefficients
pj+1=2pj-pj-1.
We easily find that the (unique) solution that satisfies the boundary conditions (2.4) is
pj=j-1N-1forj=1,2,…,N.
Assume now that μ≠0. Letting wj:=pj+1-pj.
Equation (2.3) can be rewritten as (j+c)wj=(j-c)wj-1.
Using the mathematical software program Maple, we find that the solution of this first-order difference equation that satisfies the boundary condition w1=p2 is given by
wj=-p2πsin[(2+c)π]c(c2-1)Γ2(c-1)Γ(j+1-c)Γ(j+1+c),
where Γ is the gamma function.
Next, we must solve the first-order difference equation
pj+1-pj=f(c)Γ(j+1-c)Γ(j+1+c),
where
f(c):=-p2πsin[(2+c)π]c(c2-1)Γ2(c-1),
subject to the boundary conditions (2.4). We find that, if c≠1/2, then
pj=f(c)1-2c(j+c)Γ(j+1-c)Γ(j+1+c)+f(c)c2c-1Γ(1-c)Γ(1+c)+κ,
where κ is a constant. Applying the boundary conditions (2.4), we obtain that
pj=(j+c)(Γ(j+1-c)/Γ(j+1+c))-(1+c)(Γ(2-c)/Γ(2+c))(N+c)(Γ(N+1-c)/Γ(N+1+c))-(1+c)(Γ(2-c)/Γ(2+c))forj=1,2,…,N.
Remark 2.1.
When c tends to 1/2, the solution becomes
pj=Ψ(j+1/2)-2+γ+2ln2Ψ(N+1/2)-2+γ+2ln2,
where γ is Euler's constant and Ψ is the digamma function defined by
Ψ(z)=Γ′(z)Γ(z).
Notice that
Ψ(3/2)=2-γ-2ln2,
so that we indeed have p1=0, and the solution (2.14) can be rewritten as
pj=Ψ(j+1/2)-Ψ(3/2)Ψ(N+1/2)-Ψ(3/2)forj=1,2,…,N.
Now, in the general case when Δx>0, we must solve the difference equation
pj=12A{(1+jΔx)2σ2+(1+jΔx)μΔx}pj+1+12A{(1+jΔx)2σ2-(1+jΔx)μΔx}pj-1+(1-1A(1+jΔx)2σ2)pj,
which can be simplified to
2(1+jΔx)pj=[(1+jΔx)+cΔx]pj+1+[(1+jΔx)-cΔx]pj-1.
The boundary conditions become
p0=0,pk=1.
When μ=0 (which implies that c=0), the difference equation above reduces to the same one as when Δx=1, namely (2.5). The solution is
pj=jkforj=0,1,…,k.
Writing
n=1+jΔx
and using the fact that (by hypothesis) N=1+kΔx, we obtain thatpn=n-1N-1forn=1,1+Δx,…,1+kΔx=N.
Notice that this solution does not depend on the increment Δx. Hence, if we let Δx decrease to zero and k tend to infinity in such a way that 1+kΔx remains equal to N, we have that
pn⟶n-1N-1for1≤n≤N,
which is the same as the function p(x) in (1.6) when c=0/σ2=0.
Next, proceeding as above, we obtain that, if c≠1/2, the probability pj is given by pj=(1+jΔx+cΔx)(Γ((1+(j+1)Δx-cΔx)/Δx)/Γ((1+(j+1)Δx+cΔx)/Δx))-A(1+kΔx+cΔx)(Γ((1+(k+1)Δx-cΔx)/Δx)/Γ((1+(k+1)Δx+cΔx)/Δx))-A,
where 𝒜 denotes(1+cΔx)(Γ((1+Δx-cΔx)/Δx)/Γ((1+Δx+cΔx)/Δx)). In terms of n and N, this expression becomespn=(n+cΔx)(Γ((n+Δx-cΔx)/Δx)/(Γ(n+Δx+cΔx)/Δx))-A(N+cΔx)(Γ((N+Δx-cΔx)/Δx))/(Γ((N+Δx+cΔx)/Δx))-A
for n∈{1,1+Δx,…,1+kΔx=N}. The solution reduces to
pn=Ψ((2n+Δx)/2Δx)-Ψ((2+Δx)/2Δx)Ψ((2N+Δx)/2Δx)-Ψ((2+Δx)/2Δx)ifc=1/2.
We can now state the following proposition.
Proposition 2.2.
Let n=1+jΔx for j∈{0,1,…,k}, with k such that 1+kΔx=N. The probability pn that the discrete-time Markov chain defined in Section 1, starting from n, will hit N before 1 is given by (2.23) if μ=0, and by (2.26) if c=μ/σ2≠0. The value of pn tends to the function in (2.27) when μ/σ2 tends to 1/2.
To complete this section, we will consider the case when Δx decreases to zero. We have already mentioned that when c=0, the probability pn does not depend on Δx, and it corresponds to the function p(x) in (1.6) with c=0.
Next, when c=1/2, making use of the formula
Ψ(z)~lnzforzlarge,we can write that limΔx↓0pn=limΔx↓0ln(2n+Δx)-ln(2+Δx)ln(2N+Δx)-ln(2+Δx)=lnnlnNforn∈[1,N].Again, this expression corresponds to the function p(x) given in (1.7), obtained when c=1/2.
Finally, we have: Γ(z+a)Γ(z+b)∝za-b(1+O(1z))as |z| tends to infinity (if |Arg(z+a)|<π). Hence, in the case when c≠0,1/2, we can write that limΔx↓0pn=limΔx↓0(n+cΔx)(n+Δx)-2c-(1+cΔx)(1+Δx)-2c(N+cΔx)(N+Δx)-2c-(1+cΔx)(1+Δx)-2c=n1-2c-1N1-2c-1for 1≤n≤N. Therefore, we retrieve the formula for p(x) in (1.6).
In the next section, we will derive the formulas that correspond to the function m(x) in Section 1.
3. Computation of the Mean Number of Transitions dj Needed to End the Game
As in Section 2, we will first assume that Δx=1. Then, with n=1+j for j=0,1,…,k (and 1+k=N), the function dn satisfies the following second-order, linear, nonhomogeneous difference equation: dn=pn,n+1dn+1+pn,n-1dn-1+pn,ndn+1forn=2,…,N-1.
The boundary conditions ared1=dN=0.
We find that the difference equation can be rewritten as
(n+c)dn+1-2ndn+(n-c)dn-1=-2Anσ2.
Let us now assume that μ=0, so that we must solve the second-order, linear, nonhomogeneous difference equation with constant coefficients dn+1-2dn+dn-1=-2An2σ2.With the help of the mathematical software program Maple, we find that the unique solution that satisfies the boundary conditions (3.2) is
dn=-n-1N-12Aσ2{Ψ(N)+NΨ(1,N)-(1-γ)-N(-1+π26)}+2Aσ2{Ψ(n)+nΨ(1,n)-(1-γ)-n(-1+π26)},
where
Ψ(1,x):=ddxΨ(x)
is the first polygamma function.
Next, in the general case Δx>0, we must solve (with c=0)
dj+1-2dj+dj-1=-2A(1+jΔx)2σ2
for j=0,1,…,k. The solution that satisfies the boundary conditions
d0=dk=0
is given by
dj=-jk2Aσ2(Δx)3{(1+kΔx)Ψ(1,1+kΔxΔx)+ΔxΨ(1+kΔxΔx)-Ψ(1,1Δx)-ΔxΨ(1Δx)}+2Aσ2(Δx)3{(1+jΔx)Ψ(1,1+jΔxΔx)+ΔxΨ(1+jΔxΔx)-Ψ(1,1Δx)-ΔxΨ(1Δx)}.
In terms of n:=1+jΔx and N=1+kΔx, this expression becomes dn=-n-1N-12Aσ2(Δx)3{NΨ(1,NΔx)+ΔxΨ(NΔx)-Ψ(1,1Δx)-ΔxΨ(1Δx)}+2Aσ2(Δx)3{nΨ(1,nΔx)+ΔxΨ(nΔx)-Ψ(1,1Δx)-ΔxΨ(1Δx)}.
Finally, the mean duration of the game is obtained by multiplying dn by Δt. Making use of the fact that (see (1.14)) Δt=(Δx)2/A, we obtain the following proposition.
Proposition 3.1.
When Δx>0 and μ=0, the mean duration Dn of the game is given by
Dn=-n-1N-12σ2Δx{NΨ(1,NΔx)+ΔxΨ(NΔx)-Ψ(1,1Δx)-ΔxΨ(1Δx)}+2σ2Δx{nΨ(1,nΔx)+ΔxΨ(nΔx)-Ψ(1,1Δx)-ΔxΨ(1Δx)}
for n=1,1+Δx,…,1+kΔx=N.
Next, using the fact thatΨ(x)~lnx,Ψ(1,x)~1xforxlarge,
we obtain that, as Δx decreases to zero and 1+kΔx remains equal to N,
Dn⟶2σ2{-(n-1N-1)lnN+lnn}forn∈[1,N].Notice that Dn indeed corresponds to the function m(x) given in (1.11) if c=0.
To complete our work, we need to find the value of the mean number of transitions dj in the case when μ≠0 and Δx>0. To do so, we must solve the nonhomogeneous difference equation with nonconstant coefficients (3.3). We can obtain the general solution to the corresponding homogeneous equation. However, we then need to find a particular solution to the nonhomogeneous equation. This entails evaluating a difficult sum. Instead, we will use the fact that we know how to compute dj when μ=0.
Let us go back to the geometric Brownian motion {X(t),t≥0} defined in (1.1), and let us define, for c≠1/2,
Y(t)=[X(t)]1-2c.Then, we find (see Karlin and Taylor [7, page 173]) that {Y(t),t≥0} remains a geometric Brownian motion, with infinitesimal variance σY2=(1-2c)2σ2y2, but with infinitesimal mean μY=0. In the case when c=1/2, we define
Y(t)=ln[X(t)],
and we obtain that {Y(t),t≥0} is a Wiener process with μY=0 and σY2=σ2.
Remark 3.2.
When c=1/2, we find that {X(t),t≥0} can be expressed as the exponential of a Wiener process {W(t),t≥0} having infinitesimal mean μW=0 and infinitesimal variance σW2=σ2.
When we make the transformation Y(t)=[X(t)]1-2c, the interval [1,N] becomes [1,N1-2c], respectively [N1-2c,1], if c<1/2, respectively c>1/2. Assume first that c<1/2. We have (see (1.2))
τ(x)=inf{t>0:Y(t)=1orN1-2c∣Y(0)=x1-2c}.
Now, we consider the discrete-time Markov chain with state space {1,1+Δx,…,1+kΔx=N1-2c} and transition probabilities given by (1.13). Proceeding as above, we obtain the expression in (3.9) for the mean number of transitions dj from state 1+jΔx. This time, we replace 1+jΔx by n1-2c and 1+kΔx by N1-2c, so that
dn=-n1-2c-1N1-2c-12Aσ2(Δx)3{N1-2cΨ(1,N1-2cΔx)+ΔxΨ(N1-2cΔx)-Ψ(1,1Δx)-ΔxΨ(1Δx)}+2Aσ2(Δx)3{n1-2cΨ(1,n1-2cΔx)+ΔxΨ(n1-2cΔx)-Ψ(1,1Δx)-ΔxΨ(1Δx)}
for n=1,(1+Δx)1/(1-2c),…,(1+kΔx)1/(1-2c)=N.
Assume that each displacement takes
Δt=(Δx)2(1-2c)2A
time units. Taking the limit as Δx decreases to zero (and k→∞), we obtain (making use of the formulas in (3.12)) that
Dn⟶2(1-2c)σ2{-(n1-2c-1N1-2c-1)lnN+lnn}forn∈[1,N].
This formula corresponds to the function m(x) in (1.11) when c<1/2.
When c>1/2, we consider the Markov chain having state space {1N2c-1=11+kΔx,11+(k-1)Δx,…,11+Δx,1}
(and transition probabilities given by (1.13)). To obtain dj, we must again solve the difference equation (3.7), subject to the boundary conditions d0=dk=0. However, once we have obtained the solution, we must now replace 1+jΔx by (1+jΔx)-1 (and 1+kΔx by (1+kΔx)-1). Moreover, because
j=(1+jΔx)-1Δx,
we replace j by 1/(1+jΔx)-1Δx=-j1+jΔx
(and similarly for k).
Remark 3.3.
The quantity dj here actually represents the mean number of steps needed to end the game when the Markov chain starts from state 1/(1+jΔx), with j∈{0,…,k}.
We obtain that dj=-j1+jΔx1+kΔxk2Aσ2(Δx)3{11+kΔxΨ(1,1(1+kΔx)Δx)+ΔxΨ(1(1+kΔx)Δx)-Ψ(1,1Δx)-ΔxΨ(1Δx)}+2Aσ2(Δx)3{11+jΔxΨ(1,1(1+jΔx)Δx)+ΔxΨ(1(1+jΔx)Δx)-Ψ(1,1Δx)-ΔxΨ(1Δx)}.
Next, since N2c-1=1+kΔx, setting n2c-1=1+jΔx we deduce from the previous expression thatdn=-n2c-1-1n2c-1N2c-1N2c-1-12Aσ2(Δx)3{1N2c-1Ψ(1,N1-2cΔx)+ΔxΨ(N2c-1Δx)-Ψ(1,1Δx)-ΔxΨ(1Δx)}+2Aσ2(Δx)3{1n2c-1Ψ(1,n1-2cΔx)+ΔxΨ(n2c-1Δx)-Ψ(1,1Δx)-ΔxΨ(1Δx)}
for n∈{1,(1+Δx)1/(2c-1),…,(1+kΔx)1/(2c-1)=N}.
Finally, if we assume, as above, that each step of the Markov chain takes Δt=(Δx)2(2c-1)2A
time units, we find that, when Δx decreases to zero, the mean duration of the game tends to Dn=2(2c-1)σ2{lnn-(1-n1-2c1-N1-2c)lnN}forn∈[1,N].
This last expression is equivalent to the formula for m(x) in (1.11) when c>1/2.
Remark 3.4.
Actually, the formula for m(x) is the same whether c<1/2 or c>1/2.
At last, in the case when c=1/2, we consider the random walk with state space {0,Δx,…,kΔx=lnN} and transition probabilitiespjΔx,(j+1)Δx=pjΔx,(j-1)Δx=σ22A,pjΔx,jΔx=1-σ2A.
Then, we must solve the nonhomogeneous difference equation dj+1-2dj+dj-1=-2Aσ2,
subject to the boundary conditions d0=dk=0. We find that dj=-jkAσ2k(1-k)+Aσ2j(1-j).
With lnn:=jΔx and lnN=kΔx, we get that
dn=Aσ2(Δx)2{lnn(lnN-lnn)}
for n∈{1,eΔx,…,ekΔx=N}. Assuming that Δt=(Δx)2/A, we deduce at once that, as Δx decreases to zero,
Dn⟶1σ2{lnn(lnN-lnn)}forn∈[1,N].
Thus, we retrieve the formula (1.12) for m(x) when c=1/2.
We can now state the following proposition.
Proposition 3.5.
If the state space of the Markov chain is
{1,(1+Δx)1/(1-2c),…,(1+kΔx)1/(1-2c)=N},respectively,
{1,(1+Δx)1/(2c-1),…,(1+kΔx)1/(2c-1)=N},
where c<1/2, respectively c>1/2, and the transition probabilities are those in (1.13), then the value of the mean number of steps dn needed to end the game is given by (3.17), respectively, (3.24). If n∈{1,eΔx,…,ekΔx=N} and the transition probabilities are the ones in (3.27), then the value of dn is given by (3.30).
4. Concluding Remarks
We have obtained explicit and exact formulas for the quantities pj and dj defined respectively in (1.17) and (1.18) for various discrete-time Markov chains that converge, at least in a finite interval, to a geometric Brownian motion. In the case of the probability pj of hitting the boundary N before 1, because the appropriate difference equation is homogeneous, we were able to compute this probability for any value of c=μ/σ2 by considering a Markov chain with state space {1,(1+Δx),…,(1+kΔx)=N}. However, to obtain dj we first solved the appropriate difference equation when c=0. Then, making use of the formula that we obtained, we were able to deduce the solution for any c∈ℝ by considering a Markov chain that converges to a transformation of the geometric Brownian motion. The transformed process was a geometric Brownian motion with μ=0 (if c≠1/2), or a Wiener process with μ=0 (if c=1/2). In each case, we showed that the expression that we derived tends to the corresponding quantity for the geometric Brownian motion. In the case of the mean duration of the game, the time increment Δt had to be chosen suitably.
As is well known, the geometric Brownian motion is a very important model in financial mathematics, in particular. In practice, stock or commodity prices vary discretely over time. Therefore, it is interesting to derive formulas for pj and dj for Markov chains that are as close as we want to the diffusion process.
Now that we have computed explicitly the value of pj and dj for Markov chains having transition probabilities that involve parameters μ and σ2 that are the same for all the states, we could consider asymmetric Markov chains. For example, at first the state space could be {1,…,N1,…,N2}, and we could have
μ={μ1ifn∈{1,…,N1-1},μ2ifn∈{N1+1,…,N2}
(and similarly for σ2). When the Markov chain hits N1, it goes to N1+1, respectively N1-1, with probability p0, respectively 1-p0. By increasing the state space to {1,1+Δx,…,1+k1Δx=N1,…,1+k2Δx=N2}, and taking the limit as Δx decreases to zero (with k1 and k2 going to infinity appropriately), we would obtain the quantities that correspond to pj and dj for an asymmetric geometric Brownian motion. The possibly different values of σ2 depending on the state n of the Markov chain reflect the fact that volatility is likely to depend on the price of the stock or the commodity.
Finally, we could try to derive the formulas for pj and dj for other discrete-time Markov chains that converge to important one-dimensional diffusion processes.
LefebvreM.2007New York, NY, USASpringerCoxD. R.MillerH. D.1965London, UKMethuenLefebvreM.First passage problems for asymmetric Wiener processes200643117518410.1239/jap/1143936251AbundoM.First-passage problems for asymmetric diffusions and skew-diffusion processes200916432535010.1142/S1230161209000256LefebvreM.GuilbaultJ.-L.First hitting place probabilities for a discrete version of the Ornstein-Uhlenbeck process200920091290983510.1155/2009/909835GuilbaultJ.-L.LefebvreM.On a non-homogeneous difference equation from probability theory2009438190KarlinS.TaylorH. M.1981New York, NY, USAAcademic Press