Maximum Principle of Discrete Stochastic Control System Driven by Both Fractional Noise and White Noise

In this paper, we investigate the necessary optimality conditions of the discrete stochastic optimal control problems driven by both fractional noise and white noise. Here, the admissible control region is not necessarily convex. The corresponding variational inequalities are obtained by applying the classical variation method and Malliavin calculus. We also apply the stochastic maximum principle to a linear-quadratic optimal control problem to illustrate the main result.


Introduction
We consider a stochastic control problem for state process driven by both general white noise and fractional noise with Hurst parameter H ∈ ((1/2), 1). More precisely, the state of the system is described as the following stochastic difference equation: x t k+1 � f x t k , v t k + b x t k , v t k g t k + σ x t k , v t k ω t k , k � 0, 1, . . . , N − 1, where the functions f, b, and σ and control variables u t k are introduced in Section 2. e cost functional is defined as follows: where the functions l and h are also introduced in Section 2.
Optimal control problems have a variety of applications in fields such as engineering, financial mathematics, and physics. e maximum principle is one of the main contents of modern control theory. As a necessary condition of the deterministic optimal control, it was formulated by Pontryagin and his group [1]. It states that any optimal control along with the optimal state trajectory must solve a Hamiltonian system, which is a two-point boundary value problem, plus a maximum condition of Hamiltonian. e theory was then developed extensively, and different versions of the maximum principle were derived.
With the development of the optimal control theory, some researchers began to work on the discrete case by following the Pontryagin maximum principle for continuous optimal control problems. However, the fact has been verified that the discrete case was unlike the continuous case. By imposing convexity requirement, some researchers [2,3] gave a derivation of the discrete maximum principle. A discrete optimization problem without assumptions of convexity and smoothness was shown by Mardanov et al. [4]. Taking into account the specific character of the discrete system, they obtained a necessary optimality condition which is not formulated in terms of the Hamilton-Pontryagin function.
Naturally, with the emergence of stochastic problems, more and more researchers extend the maximum principle to the stochastic case. Kushner [5] employed the spike variation and Neustadt's variational principle [6] to derive a stochastic maximum principle. Based on Girsanov transformations, Haussmann [7] extensively investigated the necessary conditions of stochastic optimal state feedback controls for systems with nondegenerate diffusion coefficients. Bismut [8] derived the adjoint equation via the martingale representation theorem. Peng [9] first considered the second-order term in the "Taylor expansion" of the variation and obtained a stochastic maximum principle for systems that are possibly degenerate, with control-dependent diffusions and not necessarily convex regions. e form of his maximum principle is quite different from the deterministic ones and reflects the stochastic nature of the problem. With the development of the fractional calculus, Han et al. [10] obtained a maximum principle for the stochastic control problem of general controlled stochastic differential systems driven by fractional Brownian motions (of Hurst parameter H > (1/2)), and the maximum principle involves Malliavin derivatives.
However, as far as the discrete stochastic system, some results for the maximum principle are analogous to the deterministic systems, which are based on the Lagrange multiplier method [11]. Recently, Lin and Zhang [12] developed a maximum principle for optimal control of discrete-time stochastic systems, and the admissible control region was nonconvex. It can be found that, up to now, the existing results appearing on the stochastic discrete version mostly study the systems with general multiplicative noise. Inspiring from this, we study the maximum principle of discrete stochastic systems driven by both fractional noise and white noise by using the classical variational method and Malliavin calculus. e admissible control region is nonconvex. e rest of this paper is organized as follows. In Section 2, we introduce some preliminaries and main assumptions needed to study the discrete stochastic control problem driven by both fractional noise and white noise. In Section 3, we derive the necessary conditions that the optimal control should satisfy. In Section 4, an example is given to illustrate the main results. In Section 5, we summarize the methods used and the results obtained.

Preliminaries
with and define Recall the operators B * H,T (see Eq. (5.35) of Hu [13]): where Let ξ 1 , ξ 2 , . . . , ξ k , . . . be an orthonormal basis of where f is a polynomial of n variables. If F is of the above form, then its Malliavin derivative D s F is defined as For any F ∈ p, we denote the following norm: Let D 1,p denote the Banach space obtained by completing p under the norm ‖ · ‖ 1,p . Let where g is a polynomial of n variables. We define its Malliavin derivative by 2 Discrete Dynamics in Nature and Society Similarly, we define ‖ · ‖ H,1,p and D H,1,p . e following duality formula will be used later to solve stochastic optimal control problems (15), (16), and (19). Lemma 1 (Theorem 6.23 of [13]). Let f : [0, T] ⊗ (Ω, F, P H ) ⟶ R be jointly measurable, and let G ∈ D H,1,2 . en, Let (Ω, F, P) be a given complete probability space. Let Let ω t k � W t k+1 − W t k be the white noise and g t k � B H t k+1 − B H t k be the fractional noise.
Let D be a bounded domain, and the space of admissible controls is defined as For the arbitrary bounded random variable v and suf- where δ km is the Kronecker delta, i.e., δ km � 1 when k � m and δ km � 0 when k ≠ m. e controlled stochastic system is described as the following discrete stochastic difference equation: Assume that f, b, σ, l, and h satisfy the following conditions: According to equation (15) and the definition of admissible controls, u and x ∈ R n . Here, all derivations are for vectors. We have Discrete Dynamics in Nature and Society where f x , f u ∈ R n×n and b x , σ x , b u , σ u ∈ R n×d . Our stochastic optimal control problem is to minimize the cost functional J(v) over, namely, to find the optimal u * ∈ U ad satisfying

The Maximum Principle
We have the following theorem as the main result of this paper. (H1-H3) hold. If (x * t k , u * t k ) is a solution to optimal control problems (15), (16), and (19) and satisfies the following equation,

Theorem 1. Let the assumptions
where f x , b x , and σ x are defined above, then there exists the following general maximum principle: Under assumption (H2), we have the following moment inequality: In fact, e estimation of the variational equation of the state equation is a critical point to obtain the maximum principle, and we need to solve the above stochastic difference equation.

Lemma 3.
There exists a unique bounded solution to the following linear matrix-valued stochastic difference equations: ϕ t k+1 � ϕ t k f x x * t k , u * t k + ϕ t k b x x * t k , u * t k g t k + ϕ t k σ x x * t k , u * t k ω t k , ϕ t 0 � I n×n .

⎧ ⎨ ⎩ (33)
Proof. For the uniqueness of the solution to (33), assume that there exists another solution ϕ ′ . We have When k � 1, we have E[|ϕ t 1 − ϕ t 1 ′ | 2 ] � 0. By the inductive method, the uniqueness of the solution to the equations is obtained. For the boundedness of the solution, it is easy to get