Optimal Stochastic Control Problem for General Linear Dynamical Systems in Neuroscience

1Big Data Research Center, Hunan University of Commerce, Changsha 410205, China 2Key Laboratory of High Performance Computing and Stochastic Information Processing (HPCSIP) (Ministry of Education of China), College of Mathematics and Computer Science, Hunan Normal University, Changsha 410081, China 3School of Business Administration, Hunan University, Changsha 410081, China 4School of Finance, Guangdong University of Foreign Studies, Guangzhou 510006, China


Introduction
The effective control of neuronal activity is one of the most exciting topics in theoretical neuroscience, with great potential for applications in healthcare.Nowadays, the application of stochastic control methods in neuroscience is becoming a significant portion of the mainstream research.Among many researches, for example, we refer to Holden (1976) for the models of the stochastic activity of neural aggregates, Iolov et al. [1] with respect to the optimal control of single neuron spike trains, and Roberts et al. [2] with respect to the review of the application of the stochastic models of brain activity.
In this paper, we study trajectory planning and control in human arm movements.When a hand is moved to a target, the central nervous system must select one specific trajectory among an infinite number of possible trajectories that lead to the target position.The content of this paper includes two parts: the first part is modeling the activities incorporating stochastic process, and the second part is quantifying task goals as cost functions and applying the sophisticated tools of optimal control theory to obtain the optimal behavior.Feng et al. [3] reviewed two optimal control problems at a different levels, neuronal activity control and movement control.They also derived the optimal signals for these two control problems.Li et al. [4] considered the robust control of human arm movements.Based on the fuzzy interpolation of an nonlinear stochastic arm system, they simplified the complex noise-tolerant robust control of the human arm tracking problem by solving a set of linear matrix inequalities using Newton's iterative method via an interior point scheme for convex optimization.Singh et al. [5] modeled reaching movements in the presence of obstacles and solved a stochastic optimal control problem that consists of probabilistic collision avoidance constraints and a cost function that trades off between effort and end-state variance in the presence of a signal-dependent noise.For more details, we refer the reader to Campos and Calado [6], Berret et al. [7], and Mainprice et al. [8].

Advances in Mathematical Physics
Yet, all the above studies discussed 1-dimensional or lower-dimension space, and the neuronal activity or movement trajectory would be involved in a higher dimension space.In this paper, motivated by Feng et al. [3], we consider a stochastic control problem for arm movement within the framework of -dimensional control space.Applying stochastic control theory, we solved the optimization problem explicitly and obtained the exact solution of the optimal trajectory, velocity, and the optimal variance.
The remainder of this paper is organized as follows.Section 2 introduces the basic model setup of the highorder linear stochastic dynamical systems for movement trajectory.In Section 3, we derive the explicit expressions for the optimal trajectory, velocity, and variance.In Section 4, we provided a 3-dimensional optimization example, and concluding remarks are given in Section 5.

Model Setup
2.1.The Integrate-and-Fire Model.In this subsection, we give the classical I & F (integrated-and-fire) model followed by Feng et al. [3], which describe the neuron activity.

𝑑𝐾 (𝑡
with (0) =  rest <  thres and where ] is the decay time constant.The synaptic input current is with   =   (),  ≥ 0 and   =   (),  ≥ 0 as Poisson processes with rates  , () and  , (),  > 0 and  > 0 being the magnitude of each excitatory postsynaptic potential (EPSP) and inhibitory postsynaptic potential (IPSP); a cell receives excitatory postsynaptic potentials (EPSPs) at  synapses and inhibitory postsynaptic potentials (IPSPs) at  inhibitory synapses.Once () crosses  thre from below, a spike is generated and  is reset to  rest .This model is termed as the IF model.
Let  rest = 0,  = ,  = , and use the usual approximation to approximate the IF models (see Feng et al. [3] and Zhang and Feng [9]); then (1) can rewriten as where {()} ≥0 is a standard Brownian motion,  > 0 is a constant, and if  = 0.5, it implies that the input is derived from a Poisson process.If  > 0.5, it is the so-called the supra-Poisson inputs, and the other is the so-called sub-Poisson inputs if  < 0.5.In addition, a larger  leads to more randomness for the synaptic inputs.

General Linear Stochastic Differential Equation.
In this subsection, we extend the one-dimensional I & F model (3) to -dimensional stochastic differential equations in which the solution process enters linearly.Such processes arise in estimation and control of linear systems, in systems, in economics, and in various other fields (see Liu [10]), as where  is an -dimensional Brown motion independent of the -dimensional initial vector , and  × ,  × 1, and  ×  matrices (), Λ(), and Σ() are nonrandom, measurable, and locally bounded, respectively.Now we define an  ×  matrix function Φ(), satisfying the following matrix differential equation: where  is the  ×  identity matrix.We know that (3) has unique (absolutely continuous) solution defined for 0 ≤  < ∞, and, for each  ≥ 0, the matrix Φ() is nonsingular.By Itô's rule, it is easily verified that solves ( 4).We suppose that ‖‖ 2 < ∞ and introduce the mean vector and covariance matrix functions as follows: () =  (, ) .(7) From (4), we can show that hold for every 0 ≤ ,  < ∞.In particular, () and () solve the linear equations:
Optimization Problem.For simplicity of notations, we let  = 0.For a point   ∈ R 1 (in fact,   expresses the arriving position component of the trajectory in the direction of component ()) and two positive numbers , , we intend to find a control signal  * () which satisfies the constrained condition: and such that the variance of  1 () arrives the minimum in [,  + ]; that is, Let Φ() = (  ) × ; by (6), we have, for 0 ≤  < ∞, Therefore, by ( 15) and (17), we have By the calculation of matrix Φ(), we easily get the following results.( In particular, for  ≥ 2, Proof.Since Φ  () = Φ(), by the definition of  (see ( 14)) and the multiplication of matrices, we get (19) at once.Since Φ(0) = , Φ  (0) =   , it is easy to get (20).
Note. if  has multiplicity  > 1 as an eigenvalue of , and  is diagonal matrix, we can also choose  independent functions with the form   exp(),  = 0, 1, . . .,  − 1.In this case, we can obtain the same result as that in the Theorem 2 by using the similar approach.By (17), it is easily seen that To this end, let us define the control signal set: where  1 ,  2 , . . .,   are the unique solution of the following (by the result of Theorem 2): The similar equations are true for the other components in -dimensional space.
From the results above, we can obtain the following conclusions.
Theorem 3.Under the optimal control framework as we set up here and  > 1/2, the optimal mean trajectory is a straight line.When  ≤ 1/2 the optimal control problem is degenerate; that is, the optimal control signal is a delta function, and ( 29) with Λ = Λ * gives us an exact relationship between time  and variance.
Proof.This proof is similar to that of Theorem 1 in Feng et al. [3]; we omit it.
Remark 4. When  = 1, the results of Theorem 3 are consistent with Feng et al. [3].The finding is also in agreement with the experimental Fitts law (see Fitts [11]); that is, the longer time of a reaching movement, the higher the accuracy of arriving at the target point.
For a point D = (  ,   ,   )  ∈  3 and two positive numbers , , we intend to find a control signal Λ() which satisfies where Λ ∈ L 2 [0, +] means that each component of it is in L 2 [0, +].To stabilize the hand, we further require that the hand will stay at  for a while, that is, in time interval [,  + ], which also naturally requires that the velocity should be zero at the end of movement.The physical meaning of the problem we considered here is clear; at time , the hand will reach the position  (see (38)), as precisely as possible (see (39)).Without loss of generality, we assume that   > 0,   > 0, and   > 0.
To use the results in the previous section, we can rewrite the optimal control problem posed in the previous paragraph as a 2-order linear stochastic dynamical system in 2dimensional space, that is, The similar equation holds true for () and ().
If we let V  () express the moving velocity in the direction of -coordinate, (40) becomes the following 2-order linear SDE: Comparing (12), it is easy to know that Since where thus by calculating, we know that where Hence, by ( 8), we get that ) . (50)

Conclusion
The experimental study of movement in human has shown that voluntary reaching movements obey Fitts law: the longer the time taken for a reaching movement, the greater the accuracy for the hand to arrive at the end point.In this paper, we study a stochastic control problem for a reaching movement within a -dimensional space.We solve this stochastic control problem explicitly and obtain the analytical solutions for optimal signals, optimal velocity, and optimal variance.Furthermore, we find that the optimal control is also consistent with Fitts law.This implies that the straight line trajectory is a natural consequence of optimal stochastic control principles, under a nondegenerate optimal control signal.