The study of the multidimensional stochastic processes involves complex computations in intricate functional spaces. In particular, the diffusion processes, which include the practically important Gauss-Markov processes, are ordinarily defined through the theory of stochastic integration. Here, inspired by the Lévy-Ciesielski construction of the Wiener process, we propose an alternative representation of multidimensional Gauss-Markov processes as expansions on well-chosen Schauder bases, with independent random coefficients of normal law with zero mean and unit variance. We thereby offer a natural multiresolution description of the Gauss-Markov processes as limits of finite-dimensional partial sums of the expansion, that are strongly almost-surely convergent. Moreover, such finite-dimensional random processes constitute an optimal approximation of the process, in the sense of minimizing the associated Dirichlet energy under interpolating constraints. This approach allows for a simpler treatment of problems in many applied and theoretical fields, and we provide a short overview of applications we are currently developing.
Intuitively, multidimensional continuous stochastic processes are easily conceived as solutions to randomly perturbed differential equations of the form
To overcome the difficulty of working in complex multidimensional spaces, it would be advantageous to have a discrete construction of a continuous stochastic process as finite-dimensional distributions. Since we put emphasis on the description of the sample paths space, at stake is to write a process
The Lévy-Ciesielski construction of the
It is also important for our purpose to realize that the Schauder elements
In view of this, we propose a construction of the multidimensional Gaussian Markov processes using a multiresolution Schauder basis of functions. As for the Lévy-Ciesielski construction, and in contrast with Karhunen-Loève decomposition, our basis is not made of orthogonal functions but the elements are of nested compact support and the random coefficients
The ideas underlying this work can be directly traced back to the original work of Lévy. Here, we intend to develop a self-contained Schauder dual framework to further the description of multidimensional Gauss-Markov processes, and, in doing so, we extend some well-known results of interpolation theory in signal processing [
In order to provide a discrete multiresolution description of the Gauss-Markov processes, we first establish basic results about the law of the Gauss-Markov bridges in the multidimensional setting. We then use them to infer the candidate expressions for our desired bases of functions, while imposing its elements to be compactly supported on nested sequence segments. Throughout this paper, we are working in a complete probability space
After recalling the definition of the multidimensional Gauss-Markov processes in terms of stochastic integral, we use the well-known conditioning formula for the Gaussian vectors to characterize the law of the Gauss-Markov bridge processes.
Let
Note that the processes considered in this paper are defined on the time interval
As stated in the introduction, we aim at defining a multiresolution description of Gauss-Markov processes. Such a description can be seen as a multiresolution interpolation of the process that is getting increasingly finer. This principle, in addition to the Markov property, prescribes to characterize the law of the corresponding Gauss-Markov bridge, that is, the Gauss-Markov process under consideration, conditioned on its initial and final values. The bridge process of the Gauss process is still a Gauss process and, for a Markov process, its law can be computed as follows.
Let
Note that the functions
Let
Note that these laws can also be computed using the expression of the density of the processes but involve more intricate calculations. An alternative approach also provides a representation of Gauss-Markov bridges with the use of integral and anticipative representation [
Recognizing the Gauss property and the Markov property as the two crucial elements for a stochastic process to be expanded to Lévy-Cesielski, our approach first proposes to exhibit bases of deterministic functions that would play the role of the Schauder bases for the Wiener process. In this regard, we first expect such functions to be continuous and compactly supported on increasingly finer supports (i.e., subintervals of the definition interval
Here, we define the nested sequence of segments that constitute the supports of the multiresolution basis. We construct such a sequence by recursively partitioning the interval
More precisely, starting from
For the sake of compactness of notations, we define
A sequence of nested intervals.
The nested structure of the supports, together with the constraint of continuity of the bases elements, implies that only a finite number of coefficients are needed to construct the exact value of the process at a given endpoint, thus providing us with an exact schema to simulate the sample values of the process on the endpoint up to an arbitrary resolution, as we will further explore.
For
The innovation process
Because of the Markovian property of the process
We deduce from the previous proposition the following fundamental theorem of this paper.
For all
The two processes
Using the expressions obtained in Proposition
Let us now define the function
We are now in position to show the following corollary of Theorem
The Gauss-Markov process
We have
We therefore identified a collection of functions
The above analysis motivates the introduction of a set of functions
To complete this program, we need to introduce some quantities that will play a crucial role in expressing the family
For all
To prove this proposition, we first establish the following simple lemma of linear algebra.
Given two invertible matrices
Directly stems from Lemma We have
Let us define
For every
The definition implies that
Before studying the property of the functions
We first introduce a family of functions
For every
Remark that the definitions make apparent the fact that these two families of functions are linked for all
Let
The space
First of all, since the functions
The proof now amounts showing the density of the family of functions we consider. Before showing this density property, we introduce for all
In fact, we show that the span of functions
We prove that
The fact that the column functions of
If
Indeed it easy to verify that, for all
From now on, abusing language, we will say that the family of
The basis
In order to exhibit a dual family of functions to the basis
Here, we make the assumptions that
We are now in a position to use the orthonormality of
As a consequence, defining the functions
The expression of the basis
For
Notice that the basis
We can now formulate the following.
Given the dual pairing in
We have to demonstrate that, for all
Otherwise, if
This proposition directly implies the main result of the section.
The collection of functions
This theorem provides us with a complementary view of stochastic processes: in addition to the standard sample paths view, this structure allows to see the Gauss-Markov processes as coefficients on the computed basis. This duality is developed in the sequel.
The Schauder basis of functions with compact supports constructed allows to define functions by considering the coefficients on this basis, which constitute sequences of real numbers in the space
After these definitions, we are in position to introduce the following useful function.
One denotes by
This sequence of partial construction applications is shown to converge to the construction application in the following.
For every
This proposition is proved in Appendix
It is important to realize that, in the multidimensional case, the space
Yet, if we additionally relax the hypothesis that
However, the situation is much simpler in the one-dimensional case: because the uniform convergence of the sample paths is preserved as long as
In the case of the
This remark does not hold that the space
We equip the space
The function
We therefore conclude that we dispose of a continuous bijection mapping the coefficients onto the sample paths,
In this section, we introduce and study the properties of the following function.
One calls
Should a function
The function
The proof of this theorem is provided in Appendix
Up to this point, we have rigorously defined the dual spaces of sample paths
Considering the infinite-dimensional subspace
The results of the preceding sections straightforwardly extend on the equivalence classes, and in particular we see that the functions
Similarly, for the matrix representation of
Denoting
The Cholesky decomposition of the finite-dimensional covariance block matrix
For every
In the finite-dimensional case, the inverse covariance or potential matrix is a well-defined quantity and we straightforwardly have the following corollary.
The Cholesky decomposition of the finite-dimensional inverse covariance matrix
The result stems for the equalities
We now show that, asymptotically, the bases
The sequence of processes
For all fixed
Now that these preliminary remarks have been made, we can evaluate, for any
For any
As
We stress the fact that the relation
The fact that this covariance is equal to the covariance of the process
The process
Our multiresolution representation of the Gauss-Markov processes appears to be the direct consequence of the fact that, because of the Markov property, the Cholesky decomposition of the finite-dimensional covariance admits a simple inductive continuous limit. More generally, triangularization of the kernel operators has been studied in depth [
We eventually underline the fact that large deviations related to this convergence can be derived through the use of the Baldi and Caramellino good rate function related to the Gaussian pinned processes [
In the following, we draw from the theory of interpolating splines to further characterize the nature of our proposed basis for the construction of the Gauss-Markov processes. Essentially adapting the results from the previous works [
In order to define the finite-dimensional sample paths as a nested sequence of RKHSs, let us first define the infinite-dimensional operator
Two straightforward remarks are worth making. First, the space
We know motivate the introduction of the Hilbert space
The Hilbert space
Consider the problem of finding all elements
From a more abstract point of view, it is well known that the covariance operator of a Gaussian measure defines an associated Hilbert structure [
In the sequel, we will use the space
We refer to such spaces as finite-dimensional approximation spaces since we remark that
The Hilbert spaces
The proof this proposition follows the exact same argument as that in the case of
The framework set in the previous section offers a new interpretation of our construction. Indeed, for all
For any
The only hurdle to prove Proposition
Writing the Gauss-Markov process
In the previous section, we have noticed that the kernel
We can now proceed to justify the main result of Proposition
The finite-dimensional processes
Proposition
The central point of this section reads as follows.
Given a function
The space
Let us now show that
When
The Dirichlet energy simply appears as the squared norm induced on
The characterization of the basis as the minimizer of such a Dirichlet energy (
Let us assume that
By Proposition
It is a simple matter of calculus to check that the expression of
Under the hypotheses of Proposition Compute the Apply the differential operator Orthonormalize the column functions Apply the integral operator
Notice finally that each of these points is easily implemented numerically.
In the one-dimensional case, the construction of the Gauss-Markov process is considerably simplified since we do not have to consider the potential degeneracy of matrix-valued functions. Indeed, in this situation, the centered Gauss-Markov process
This reads on
In the multidimensional case, the explicit expressions for the basis functions
We consider in this section that
Recognizing the
This form shows that the Schauder basis for multidimensional rotations results from the multiplication of the triangular-shaped elementary function used for the Lévy-Ciesielski construction of the Wiener process with the flow of the equation, that is, the elementary rotation.
The simplest example of this kind is the stochastic sine and cosine process corresponding to
Construction of the stochastic sine and cos Ornstein-Uhelenbeck processes for the parameters given in (
(a) Basis for the construction of the Integrated Wiener process (
In applications, it often occurs that people use smooth stochastic processes to model the integration of noisy signals. This is for instance the case of a particular subject of a Brownian forcing or of the synaptic integration of noisy inputs [
Let
Furthermore, because of the simplicity and the sparsity of the matrices involved, we can identify in a compact form all the variables used in the computation of the construction basis for these processes. In particular, the flow
Observe that, for all
We study in more detail the case of the integrated and doubly-integrated Wiener process (
Thus far, all calculations, propositions, and theorems are valid for any finite-dimensional the Gauss-Markov process and all the results are valid pathwise, that is, for each sample path. The analysis provides a Hilbert description of the processes as a series of standard Gaussian random variables multiplied by certain specific functions, that form a Schauder basis in the suitable spaces. This new description of Gauss-Markov processes provides a new way for treating problems arising in the study of stochastic processes. As examples of this, we derive the Itô formula and the Girsanov theorem from the Hilbertian viewpoint. Note that these results are equalities in law, that is, dealing with the distribution of stochastic processes, which is a weaker notion compared to the pathwise analysis. In this section, we restrict our analysis to the one-dimensional case for technical simplicity.
The closed-form expressions of the basis of functions
The integral operator
Now that we dispose of all the explicit forms of the basis functions and related operators, we are in position to complete our program and start by proving the very important Itô formula and its finite-dimensional counterpart before turning to the Girsanov theorem.
A very useful theorem in the stochastic processes theory is the Itô formula. We show here that this formula is consistent with the Hilbert framework introduced. Most of the proofs can be found in Appendix
Let
The proof of this proposition is quite technical and is provided in Appendix
Let
This theorem is proved in Appendix
The Itô formula implies in particular that the multiresolution description developed in the paper is valid for every smooth functional of a Gauss-Markov process. In particular, it allows a simple description of exponential functionals of Gaussian Markovian processes, which are of particular interest in mathematics and have many applications, in particular in economics (see, e.g., [
Therefore, we observe that in the view of the paper, Itô formula stems from the nonorthogonal projections of basis element. For multidimensional processes, the proof of the Itô formula is deduced from the one-dimensional proof and would involve the study of the multidimensional bridge formula for
We eventually remark that this section provides us with a finite-dimensional counterpart of the Itô formula for discretized processes, which has important potential applications, and further assesses the suitability of using the finite resolution representation developed in this paper. Indeed, using the framework developed in the present paper allows considering finite-resolution processes and their transformation through nonlinear smooth transformation in a way that is concordant with the standard stochastic calculus processes, since the equation on the transformed process indeed converges towards its Itô representation as the resolution increases.
In the framework we developed, transforming a process
The general problem consists therefore in studying the relationship between two real Gauss-Markov processes
We have noticed that the spaces
The previous remarks allow us to restrict without loss of generality our study to the processes defined for same function
Let us now consider that
Depending on the space we are considering (either coefficients or trajectories), we define the two following operators mapping the process The The
The operators They are linear measurable bijections For every The determinants of
The proof of these properties elementary stems from the analysis done on the functions
From the properties proved on the lift operators, we are in position to further analyze the relationship between the probability distributions of
Given the finite-dimensional measures
In the finite-dimensional case, for all
Let us now justify from a geometrical point of view why this formula is a direct consequence of the finite-dimensional change of variable formula on the model space
Since
The pathwise expression of the Radon-Nikodym derivative extends to the infinite-dimensional representation of
Considering the coefficients of the matrix representation of
Assume that
Notice that we can further simplify expression (
The operator
We prove this essential point in Appendix
We now proceed to prove the Girsanov theorem by extending the domain of the quadratic form associated with
In the infinite-dimensional case, the Radon-Nikodym derivative of
In order to demonstrate the Girsanov theorem from our geometrical point of view, we need to establish the following result.
The positive definite quadratic form on
We start by writing the finite-dimensional Radon-Nikodym derivative
The derivation of the Girsanov formula (
Observe that, if
The discrete construction we present displays both analytical and numerical interests for further applications. From the analysis viewpoint, even if the basis does not exhibit the same orthogonal properties as the Karhunen-Loève decomposition, it has the important advantage of saving the structure of sample paths through its property of strong pathwise convergence and of providing a multiscale representation of the processes, which contrasts with the convergence in the mean of the Karhunen-Loève decomposition. From the numerical viewpoint, three Haar-like properties make our decomposition particularly suitable for certain numerical computations: (i) all basis elements have compact support on an open interval that has the structure of dyadic rational endpoints, (ii) these intervals are nested and become smaller for larger indices of the basis element, and (iii) for any interval endpoint, only a finite number of basis elements are nonzero at that point. Thus the expansion in our basis, when evaluated at an interval endpoint (e.g., dyadic rational), terminates in a finite number of steps. Moreover, the very nature of the construction based on an increasingly refined description of the sample paths paves the way to coarse-graining approaches similar to wavelet decompositions in signal processing. In view of this, our framework offers promising applications. The first application we envisage concerns the problem of first-hitting times. Because of its manifold applications, finding the time when a process first exits a given region is a central question of stochastic calculus. However, closed-form theoretical results are scarce and one often has to resort to numerical algorithms [ The present study is developed for the Gauss-Markov systems. However, many models arising in applied science present nonlinearities, and in that case, the construction based on a sum of Gaussian random variables will not generalize. However, the Gaussian case treated here can nevertheless be applied to perturbation of nonlinear differential equations with small noise. Let All these instances are exemplary of how our multiresolution description of the Gauss-Markov processes offers a simple yet rigorous tool to broach a large number of open problems and promises fascinating applications both in theoretical and in applied science.
First passage time for an Ornstein-Uhlenbeck process
In the case of the primitive of the Wiener process, straightforward linear algebra computations lead to the two bases of functions
For the doubly-integrated Wiener process, the construction of the three-dimensional process involves a family of three 3-dimensional functions, which constitutes the columns of a
(a) Basis for the construction of the Doubly Integrated Wiener process (
This appendix is devoted to the proofs of the properties of the lift operator enumerated in Proposition
The operator
Eventually,
All these properties are deduced from the properties of the functions Since we have the expressions of the matrices of the finite-dimensional linear transformations, it is easy to write the linear transformation of The eigenvalues of the operator are therefore the diagonal elements From the expression of
Since, for every
Notice that, if
Similar properties are now proved for the process lift operator
The function
Moreover, for every
The function We write the linear transformation of The upper bound directly follows from the fact that Since
Note that Lemma
We have the following set of properties in terms of matrix operations
Let us write We have We have We have We have
In this appendix, we provide the proofs of the main properties used in the paper regarding the construction and the coefficient applications.
We start by addressing the case of the construction application introduced in Section
We start by proving Proposition
For the sake of simplicity, we will denotes for any function
Let
By definition of
We therefore need to upperbound the uniform norm of the function
Moreover, since
Now using this bound and (
This proposition being proved, we dispose of the map
The function
The application
In the one-dimensional case, as mentioned in the main text, because the uniform convergence of the sample paths is preserved as long as
In the one-dimensional case, the space
This point can be seen as a direct consequence of the characterization of the local Hölder exponent of a continuous real function in terms of the asymptotic behavior of its coefficients in the decomposition on the Schauder basis [
To underline that we place ourselves in the one-dimensional case, we drop the bold notations that indicate multidimensional quantities. Supposing that
We equip the space
The function
Consider an open ball
In this section of the appendix, we show some useful properties of the coefficient application introduced in Section
The function
The function To prove that Before proving the measurability of
For any
We now demonstrate Theorem
The function
Let
In this section, we provide rigorous proofs of Proposition
Let
We assume that
We define
Let
The integration by parts formula directly implies the Itô formula through a density argument as follows. Let
Let now
In this section, we demonstrate Theorem
The operator
Notice first that
We now proceed to demonstrate that
Since the kernel
In this section we provide the quite technical proof of Lemma
The positive definite quadratic form on
The proof of this lemma uses quite similar materials to those used in the proof of the Itô theorem. However, since this result is central for giving insight on the way our geometric considerations relate to the Girsanov theorem, we provide the detailed proof here.
Consider
The authors wish to thank Professor Marcelo Magnasco for many illuminating discussions. This work was partially supported by NSF Grant EF-0928723 and ERC Grant NERVI-227747.