MEAN ANISOTROPY OF HOMOGENEOUS GAUSSIAN RANDOM FIELDS AND ANISOTROPIC NORMS OF LINEAR TRANSLATION-INVARIANT OPERATORS ON MULTIDIMENSIONAL INTEGER LATTICES

Sensitivity of output of a linear operator to its input can be quantified in various ways. In Control Theory, the input is usually interpreted as disturbance and the output is to be minimized in some sense. In stochastic worst-case design settings, the disturbance is considered random with imprecisely known probability distribution. The prior set of probability measures can be chosen so as to quantify how far the disturbance deviates from the white-noise hypothesis of Linear Quadratic Gaussian control. Such deviation can be measured by the minimal Kullback-Leibler informational divergence from the Gaussian distributions with zero mean and scalar covariance matrices. The resulting anisotropy functional is defined for finite power random vectors. Originally, anisotropy was introduced for directionally generic random vectors as the relative entropy of the normalized vector with respect to the uniform distribution on the unit sphere. The associated a-anisotropic norm of a matrix is then its maximum root mean square or average energy gain with respect to finite power or directionally generic inputs whose anisotropy is bounded above by a≥0. We give a systematic comparison of the anisotropy functionals and the associated norms. These are considered for unboundedly growing fragments of homogeneous Gaussian random fields on multidimensional integer lattice to yield mean anisotropy. Correspondingly, the anisotropic norms of finite matrices are extended to bounded linear translation invariant operators over such fields.


Introduction
The sensitivity of the output of a given linear operator to its input can be quantified in many different ways.This issue is important in the situations, normally studied by Control Theory, where the input plays the role of a disturbance and it is desirable to minimize the output in some sense.In turn, this last is associated with a certain performance criterion and depends on assumptions made on the input.
For deterministic disturbances, the largest singular value of the operator can be used.In application to dynamic systems, this approach is employed by H ∞ control theory, e.g.[26,8,7,16] to mention a few.Alternatively, if the disturbance is a random vector with homoscedastic uncorrelated entries, then an appropriate measure of the sensitivity is the trace norm of the operator.This "white noise" hypothesis is the principal supposition in Wiener-Hopf-Kalman filtering and Linear Quadratic Gaussian (LQG) control theories [11,25,1,5,13].
In more realistic situations, one is confronted by statistical uncertainty where the disturbance can be considered random, but with imprecisely known probability distribution.The associated set of probability measures constitutes the prior information on the disturbance.This leads to stochastic worst-case design problems which nowadays form a wide area of research, see e.g.[21,14] and references therein.
Among various settings which are possible within the paradigm, we choose the one where the prior set of probability distributions serves to quantify how far the disturbance is expected to deviate from the white-noise hypothesis of LQG control.As a measure of such deviation we use the minimal Kullback-Leibler informational divergence [9,Chapter 5] of the probability distribution of a random vector from the Gaussian distributions with zero mean and scalar covariance matrices.
The resulting functional, called anisotropy, is well defined for absolutely continuously distributed square integrable (or briefly, finite power) random vectors.The so-defined anisotropy functional was studied in [23] and is not dissimilar to the power-entropy construct considered in [2] for scalar random variables.The sensitivity of a linear operator can then be described by its a-anisotropic norm defined as the maximum root mean square gain of the operator with respect to random vectors whose anisotropy is bounded above by a given nonnegative parameter a.The corresponding worst-case input turns out to be Gaussian distributed with zero mean.In [23] this approach was used to develop a robust performance analysis of control systems evolving on a finite discrete time horizon.
The anisotropy-based approach to quantitative description of the statistical uncertainty in entropy theoretic terms for the purposes of robust control was proposed in [19] and [22], where anisotropy of a random vector was defined in a different way, as the relative entropy of the normalized vector with respect to the uniform distribution on the unit sphere.The associated a-anisotropic norm is the maximum average energy gain with respect to directionally generic disturbances for which the normalized vector is well-defined and absolutely continuously distributed on the sphere.In [22], the anisotropy functional was also extended to stationary Gaussian sequences by computing it for increasingly long fragments of the sequence and taking an appropriate limit to obtain mean anisotropy per unit time.
The present paper is aimed at a more systematic comparison of the anisotropy functionals and anisotropic norms and at generalization of the aforementioned constructs to bounded linear translation-invariant operators over vector-valued homogeneous Gaussian random fields on multi-dimensional integer lattices.These results can find applications in robust recovery of multivariate functions by noise corrupted data, e.g. in image processing, and in robust control of flexible structures.
The paper is organized as follows.Sections 2 and 3 provide definitions and basic properties of the anisotropy functionals for the classes of directionally generic and finite power random vectors.Complementing the results of [23, Section 2.2], the functionals are compared in Section 4 where a class of quasigaussian random vectors is described for which the anisotropies share the same value.In Section 5, the anisotropies are computed for zero mean Gaussian random vectors.Section 6 gives definitions and basic properties of the anisotropic norms of matrices induced by the aforementioned anisotropy functionals.In Sections 7 and 8, the anisotropies are considered for fragments of a homogeneous Gaussian random field on a multidimensional integer lattice obtained by restricting the field to finite subsets of the lattice.In Section 8, it is shown that as the subsets tend to infinity in the sense of van Hove, widely used in Statistical Mechanics of lattice systems [15,18], the properly normalized anisotropies have a common limit, the mean anisotropy of the field.In Sections 9 and 10, the anisotropic norm is defined for bounded linear translation invariant operators over homogeneous Gaussian random fields, and formulas are given for computing the norm.In Section 11, an asymptotic connection of the norm is established with those of finite dimensional projections of the operator associated with finite subsets of the lattice.In Sections 12 and 13, proofs of the main theorems are given along with subsidiary lemmas.

Directionally Generic Random Vectors
Recall that for probability measures M and N on a common measurable space (X, E), the Kullback-Leibler informational divergence [9, p. 89] of M with respect to (wrt) N is defined as Here, E( • ) denotes expectation in the sense of M , and dM/dN : X → R + is the Radon-Nikodym derivative in the case of absolute continuity of M wrt N written as M N .By the Divergence Inequality [9, Lemma 5.2.1 on p. 90], the quantity D(M N ) is always nonnegative and is only zero if M = N .
If M and N are probability distributions of random elements ξ and η or are given by their probability density functions (pdf) f and g wrt a common dominating measure, we shall, slightly abusing notations, occasionally replace the symbols M or N in D(M N ) with ξ, f or η, g, respectively.
Definition 2.1:Say that a R m -valued random vector W , defined on an underlying probability space (Ω, F, P), is directionally generic if P(W = 0) = 0 and the probability distribution of W/|W | is absolutely continuous wrt to the uniform distribution U m on the unit sphere Denote by D m the class of m-dimensional directionally generic random vectors.Anisotropy of W ∈ D m was defined in [22] as the quantity ( Here, g = dQ/dU m is the pdf of V = W/|W | wrt U m , and Q is the probability distribution of V expressed in terms of the distribution P of W as where where ϕ is a positive scalar random variable and R ∈ so(m) is a nonrandom orthogonal (m × m)-matrix.In particular, A • (W ) is invariant wrt nonrandom permutations of the entries of W .Therefore, A • (W ) can also be interpreted as an information theoretic measure of directional nonuniformity of P , i.e. noninvariance of Q under the group of rotations.
For example, any random vector W , distributed absolutely continuously wrt the m-dimensional Lebesgue measure mes m , is directionally generic.In this case, the pdf g of W/|W | is expressed in terms of the pdf f of W as Here, R m is an absolutely continuous measure on R + defined by where  there exists a positive scalar random variable ϕ such that ϕW ∈ L m 2 .Based on a power-entropy construct considered in [2] for scalar random variables, a definition of anisotropy A(W ) of W ∈ L m 2 , alternative to (2.1), was proposed in [23] as where is the differential entropy [4, p. 229] of W , and f is its pdf wrt mes m .In (3.1), p m,λ denotes the Gaussian pdf on R m with zero mean and scalar covariance matrix λI m , In general, denote by G m (C) the class of R m -valued Gaussian distributed random vectors with zero mean and covariance matrix C. In the case det C = 0, the corresponding pdf is where x Q = x T Qx denotes the (semi-) norm of a vector x induced by a positive (semi-) definite symmetric matrix Q.The lemma below shows that the anisotropy functional (3. where the minimum is attained only at W ∈ G m (C); (c) For any W ∈ L m By Lemma 3.1(c) which essentially replicates the definition (3.1), the anisotropy A(W ) is an information theoretic distance of the probability distribution of W from the Gaussian distributions with zero mean and scalar covariance matrices.At the same time, A(W ) quantifies noninvariance of the distribution under the group of rotations.

Quasi-Gaussian Random Vectors
Denote by L + 2 the class of square integrable R + -valued random variables, distributed absolutely continuously wrt mes 1 .For any ξ ∈ L + 2 with pdf α wrt the measure R m given by (2.4)-(2.5),define the quantity where is the differential entropy of ξ wrt R m .A variational meaning of (4.
The representations (4.3) and (4.4) imply that Also note that for any W ∈ D m , A • (W ) = inf A(ϕW ) where the infimum is taken over all the positive scalar random variables ϕ such that ϕW ∈ L m 2 .

Anisotropy of Gaussian Random Vectors
Lemma 5.1: For W ∈ G m (C) with det C = 0, the anisotropies (2.1) and (3.1) satisfy the relations where ζ is a random vector distributed uniformly on the unit sphere S m .Proof: Plugging the Gaussian pdf (3.3) in (2.3) and using (2.5), obtain that the pdf of Hence, (2.1) reads Introducing the random vector Since ζ = Z/|Z| is uniformly distributed on the unit sphere S m , from (5.3) it follows that This last equality and (5.2) imply the left-most equality in (5.1).The equality on the right of (5.1) is a corollary of Lemma 3.1(b), while the inequality follows from the general relationship (4.5), concluding the proof.

Anisotropic Norms of Matrices
Let F ∈ R p×m be interpreted as a linear operator with R m -valued random input W and R p -valued output Z = F W .For any a ∈ R + , consider the a-anisotropic norms of F associated with the anisotropy functionals (2.1) and (3.1), Here, characterizes the average energy gain of F wrt W and is well-defined as soon as P(W = 0) = 0, while measures the root mean square gain of F wrt a square integrable input W . Clearly, the norms (6.1) and (6.2) are nondecreasing in a ∈ R + and satisfy where Proof:Let W be a square integrable random vector satisfying P(W = 0) = 0 and such that |W | and and consequently, by (6.3) and (6.4), In particular, (6.5) holds for any quasigaussian W ∈ Q m (see Definition 4.1).Combining this last property with (4.6), obtain that thereby concluding the proof.

Fragments of Random Fields
Denote by GF m,n (S) the class of R m -valued homogeneous Gaussian random fields W = (w x ) x∈Z n on the n-dimensional integer lattice Z n with zero mean and spectral density function (sdf) S : Ω n → C m×m , where Ω n = [−π, π) n .Since S can be extended to R n (2π)-periodically in each of its n variables, Ω n is identified with n-dimensional torus.
For any ω ∈ Ω n , the matrix S(ω) is Hermitian and positive semi-definite, and satisfies S(−ω) = (S(ω)) T .The corresponding covariance function  Under this condition, the sdf S is continuous on the torus Ω n and so also are the functions Ω n ω → λ min (S(ω)), λ max (S(ω)), with λ max ( • ) denoting the largest eigenvalue of a Hermitian matrix.In this case, the strict regularity of W is equivalent to nonsingularity of S(ω) for all ω ∈ Ω n .Denote by Z n = {X ⊂ Z n : 0 < #X < +∞} the class of nonempty finite subsets of Z n , where #( • ) stands for the counting measure.For any X ∈ Z n , the restriction of W to X is identified with the R m#X -valued Gaussian random vector The order in which the random vectors w x are "stacked" one underneath the other in (7.3) is not essential for what follows.However, to avoid ambiguity, the set X will be assumed lexicographically ordered.The spectrum of the covariance matrix is invariant under translations of X ∈ Z n since for any z ∈ Z n there exists a permutation matrix Π of order m#X such that C X+z = ΠC X Π T .If the random field W is strictly regular, then det C X > 0 for any X ∈ Z n .This implication follows from the spectral bounds ess inf where, under the assumption (7.2), ess inf and ess sup can be replaced with min and max.Applying Lemma 5.1 to (7.3) and using the identity Tr C X = Tr c 0 #X, obtain that where ζ X is a random vector, distributed uniformly on the unit sphere S m#X .It turns out that, when divided by #X, both anisotropies in (7.6) have a common limit as the set X tends to infinity in a sense specified below.

Definition of Mean Anisotropy
With every X ∈ Z n , associate the function It is worth noting that #XD X is the geometric covariogram [12, p. 22] of the set X wrt the counting measure #.Clearly, supp D X = {x − y : x, y ∈ X}.A probabilistic interpretation of D X is as follows.Let ξ X and η X be independent random vectors each distributed uniformly on X.Then the probability mass function (pmf) of θ X = ξ X − η X is expressed in terms of (8.1) as Recall that a sequence of sets X k ∈ Z n , labeled by positive integers k ∈ N, is said to tend to infinity in the sense of van Hove [18, p. 45] if lim k→+∞ D X k (z) = 1 for every z ∈ Z n .The property induces a topological filter on the class Z n and is denoted by ∞.
By the identity #X = z∈Z n D X (z) which follows from (8.2), a necessary condition for X ∞ is #X → +∞.A simple example of a sequence which tends to infinity in the sense of van Hove is provided by the discrete hypercubes Theorem 8.1: Let W ∈ GF m,n (S) be strictly regular and let its covariance function be absolutely summable.Then the asymptotic behaviour of the anisotropies (2.1) and (3.1) of the random vectors W X in (7.3) is described by 3) The proof of the theorem is given in Section 12.The common limit on the right of (8.3) will be referred to as mean anisotropy of the field W and denoted by A(W ).
Example 8.1: Compute the mean anisotropy of W ∈ GF m,n (S) with covariance function where c 0 ∈ R m×m is a positive definite symmetric matrix, and ρ 1 , . . ., ρ n are positive reals, with ρ k interpreted as a correlation radius of W along the k-th coordinate axis in R n .The corresponding sdf S is given by Here, for every 1 ≤ k ≤ n, the function σ k : Ω 1 → R + , defined by is sdf of a stationary scalar Gaussian sequence (ξ t ) t∈Z with zero mean and covariance function E(ξ t ξ 0 ) = α |t| k .Applying the Szego-Kolmogorov formula and using the Markov property of the sequence together with the Normal Correlation lemma, obtain where Var (• | •) denotes conditional variance.Clearly, the random field W satisfies the assumptions of Theorem 8.1 and, by (8.4), its mean anisotropy defined in (8.3) reads Here, the right-most sum behaves asymptotically like − n k=1 ln ρ k if the correlation radii are all large.

Anisotropic Norm of LTI Operators
Denote by r,n Here, the output where Z n x → f x ∈ R p×m is the impulse response function.The operator is identified with the transfer function F : Ω n → C p×m defined by 3) The L ∞ -norm of this last, ess sup ω∈Ω n F (ω) ∞ , coincides with (9.1) and, upon rescaling, is an upper bound for the L 2 -norm, where the map H : Ω n → C m×m is defined by The inequality on the right of (9.4) becomes an equality iff there exists λ ∈ R + such that H(ω) = λI m for mes n -almost all ω ∈ Ω n .Definition 9.1: ∞ and its input W ∈ GF m,n (S), then the convergence of the series (9.2) is understood in mean square sense and the output satisfies Z ∈ GF p,n (F SF * ).In particular, Tr (H(ω)S(ω))dω.
Recalling the relations (7.1) and E|w 0 | 2 = Tr c 0 , quantify the root mean square gain of F wrt W by For every a ≥ 0, define the a-anisotropic norm of the operator F as Here, the supremum is taken over all the strictly regular homogeneous Gaussian random fields W whose mean anisotropy (8.3) is bounded above by a. Denote by W a (F ) the corresponding set of the worst-case inputs W at which the supremum in (9.6) is attained.

Computing Anisotropic Norm
Assuming the operator F ∈ L p×m,n ∞ fixed, for notational convenience let Recalling (9.5), define the functions A, N , Φ, Ψ on Q by ) Here, N is extended to 0 by continuity as N (0) = lim q→0+ N (q) = F 2 / √ m.The functions (10.2)-(10.5)are all analytic, nondecreasing in q ∈ Q and take values in , +∞) and R + , respectively.Following the technique of a randomized singular value used in the proof of [6, Lemma 4], one verifies that if F is nonround in the sense of Definition 9.1, then A is strictly increasing and convex on the interval Q, with A(0) = 0 and A(q) → +∞ as q → F −2 ∞ −.These properties imply that the function A is invertible and its inverse A −1 : R + → Q is strictly increasing and concave.
Theorem 10.1: Let F ∈ L p×m,n ∞ be nonround.Then for any a ≥ 0, the aanisotropic norm (9.6) is expressed in terms of the functions (10.2) and (10.3) as (10.6) The corresponding set of worst-case inputs is The proof of the theorem is similar to that of [6, Theorem 3] and therefore omitted.Using the remark made after the proof of [23,Theorem 2], one verifies that the norm ∞ with impulse response where I Y : Z n → {0, 1} is the indicator function of the set Y .The corresponding transfer function is Clearly, where ξ and η are independent random vectors each distributed uniformly on Y .The corresponding pmf is given by D Y /#Y as in (8.2).Now let Θ = (θ k ) k∈N be a sequence of independent (Y − Y )-valued random vectors, each with the cf H. Associate with Θ a random walk Σ = (σ k ) k∈Z + on Z n defined by For every k ∈ Z + , H k is cf of σ k , and hence, (2π) −n Ω n (H(ω)) k dω = P(σ k = 0).Therefore, the function (10.4) takes the form

Connection with Anisotropic Norms of Matrices
A connection of the anisotropic norm of the operator F with those of finite matrices (see Section 6) is established below.To formulate the statement, for every X ∈ Z n , introduce a matrix F X ∈ R p#X×m#X by appropriately restricting the impulse response function of F , F X = block x,y∈X (f x−y ).(11.1) If P X and M X are the orthogonal projectors in p,n 2 and m,n 2 to the subspaces of signals whose support is contained in X, then P X F M X = F X M X .
Theorem 11.1: Let F ∈ L p×m,n ∞ be nonround and let its impulse response function be absolutely summable, i.e.
x∈Z n f x ∞ < +∞.Then for every a ≥ 0, the (a#X)anisotropic norms (6.1) and ( 6.2) of the matrices (11.1) have the a-anisotropic norm of F in (9.6) as their common limit, 2 The theorem is proved in Section 13.
12 Proof of Theorem 8.1 For any X ∈ Z n and r ∈ N, introduce the function E X,r : Z rn → [0, 1] which maps a vector y = (y k ) 1≤k≤r , formed by y 1 , . . ., y r ∈ Z n , to Comparison with (8.1) shows that E X,1 (z) = E X,2 (z, −z) = D X (z) for any z ∈ Z n .By (12.1), Therefore, the definition of convergence in the sense of van Hove (see Section 8) yields lim For notational convenience in the sequel, introduce the set (12.5) Here, for any y = (y k ) 1≤k≤r formed by y 1 , . . ., y r ∈ Z n , the Fourier coefficient ψ y is given by In these notations, (7.1) and (7.4) imply that for any X ∈ Z n , Hence, recalling (12.1) and (12.3), obtain By the inequality |Tr A| ≤ m A ∞ which holds for any A ∈ R m×m and by submultiplicativity of • ∞ , (12.6) implies that Consequently, the assumption of the lemma assures that x∈O r |ψ y | < +∞, thus legitimating the passage to the limit under the sum in (12.7) on a basis of (12.2), lim It now remains to note that by (12.3), (12.5) and (12.6), Tr (S(ω)) r dω which, in combination with (12.8), immediately yields (12.4), thereby completing the proof.
Note that the assertion of Lemma 12.1 for the particular case r = 2 can be established in a much simpler way.Indeed, by (7.4), (8.1) and by Parseval's equality, Lemma 12.2: Let W ∈ GF m,n (S) be strictly regular and let (7.2) hold.Then the matrices (7.4) ln det S(ω)dω (12.10) Proof:By (7.5), under the assumptions of the lemma, for any X ∈ Z n , the spectrum of C X is entirely contained in the interval ∆ = min ω∈Ω n λ min (S(ω)), max ω∈Ω n λ max (S(ω)) (12.11) which is separated from zero and bounded.Since the logarithm function is expandable on the interval to a uniformly convergent power series, application of Lemma 12.1 to the series yields (12.10).Note that the assertion of Lemma 12.2 under weaker assumptions is well-known in the case n = 1 for Toeplitz forms [10], and is closely related to Szego-Kolmogorov formula for Shannon entropy rate in stationary Gaussian sequences.For the multivariate case n > 1, it is worth pointing out the links to the mean entropy results for Gibbs-Markov random fields [18, pp. 44-47].
Lemma 12.3: Let ζ be uniformly distributed on the unit sphere S r , and let C ∈ R r×r be a positive semi-definite symmetric matrix.Then ) where Var ( • ) denotes the variance of a random variable.Moreover, if det C = 0 and r ≥ 3, then . By a straightforward calculation (also see [24, p. 179]), for all 1 ≤ j = k ≤ r, ) ) From (12.15) it follows that which coincides with (12.12).Furthermore, taking (12.16) and (12.17) into account, obtain that The equalities (12.18) and (12.19) immediately imply that which coincides with (12.13).To prove (12.14), note that, by the geometric-arithmetic mean inequality, Therefore, for any r ≥ 3, r which yields (12.14), completing the proof.Remark 12.1: Note that the C-independent multiplier on the right of (12.14) is convergent, where (ln Γ) (λ) = Γ (λ)/Γ(λ) is the logarithmic derivative of the Euler gamma function.
Proof of Theorem 8.1: For any X ∈ Z n , let ζ X be uniformly distributed on the unit sphere S m#X .Then (7.6) reads Let us show that ζ X 2 C X is mean square convergent as X ∞.Applying (12.12) of Lemma 12.3 and recalling (7.4), obtain Combining (12.13) with (12.9) yields

Var
This relation and (12.22) imply the convergence of ζ X 2 C X to Tr c 0 /m in mean square sense and consequently, in probability, Recalling that under assumptions of the theorem, the spectra of the matrices C X are all contained in (12.11) which gives (8.3), completing the proof.Remark 12.2: If the random field W is not strictly regular, then the dominated convergence argument that was used to derive (12.24) from (12.23) fails.In this case, however, the condition lim inf if it holds, implies the uniform integrability of ln ζ X 2 C X as X ∞.Indeed, by the identity exp(| ln u|) = max(u, 1/u) for any u > 0, Hence, applying (12.22), (12.14) and (12.20), obtain that lim sup 13 Proof of Theorem 11.1 Indeed, using (8.1) and (9.4), obtain that For every X ∈ Z n , associate with (11.1) a positive semi-definite symmetric matrix Tr (H(ω)) r dω, (13.4)where the map H is defined by (9.5).
Under the assumptions of Lemma 13.1, for any r ∈ N, By the well-known properties of Hölder norms, for any positive semi-definite Hermitian A ∈ C d×d , the quantity (Tr A r /d) 1/r is nondecreasing in r ∈ N and converges to λ max (A) as r → +∞.Applying the monotone convergence argument to both sides of (13.10), obtain the following refinement of (13.1), lim Proof of Theorem 11.1: Recalling (13.3), for any X ∈ Z n introduce the functions ) Φ X (q) = Tr (I m#X − qH X ) −1 m#X , ( Ψ X (q) = − ln det (I m#X − qH X ) m#X .(13.15)These are similar to (10.2)-(10.5)and well-defined on the interval which, by (13.1), contains (10.1).Comparison of (13.2) with (13.11) shows that the nonroundness of the operator F implies for all X ∈ Z n large enough in the sense of van Hove convergence to infinity.For any such X, the function A X : Q X → R + is strictly increasing and has a well-defined inverse A −1 X : R + → Q X .Application of [23,Theorem 4] to the matrix (11.1) gives that for any a ∈ R + , |||F X ||| a#X = N X (A −1 X (a#X)).(13.17)Expanding (13.14) and (13.15) to the Taylor series whose convergence is uniform in X ∈ Z n for any q ∈ Q, and using Lemma 13.1, obtain that Φ X (q) and Ψ X (q) (and so also A X (q)/#X and N X (q)) converge to Φ(q) and Ψ(q) (respectively, to A(q) and N (q)) as X ∞.Moreover, a slightly refined reasoning shows that the convergence holds for the derivatives of the functions as well and is uniform in q over any compact subset of the interval (10.1).Therefore, (a) → A −1 (a) as X ∞. (13.18)Combining the last property with (13.17This proves the right-most equality in (11.2).By Lemma 6.1, the left-most equality in (11.2) will be proved if we show that lim inf .20)For this purpose, consider a random vector W X,a ∈ G m#X (Σ X,a ) where Σ X,a = (I m#X − q X,a H X ) −1 , q X,a = A −1 X (a#X).(13.21)By [23,Theorem 4], W X,a is a worst-case input for F X in the sense of the (a#X)anisotropic norm (6.2) associated with the root mean square gain (6.4).That is, |||F X ||| a#X = N(F X , W X,a ), A(W X,a ) = a#X.
Applying to W X,a the inequality (4.5), obtain that A • (W X,a ) ≤ a#X.Consequently, the average energy gain of F X wrt W X,a provides a lower bound for the (a#X)anisotropic norm of F X defined by (6.1), i.e.
|||F X ||| a#X, • ≥ N • (F X , W X,a ).(13.22)To compute the gain, write W X,a = Σ 1/2 X,a V X where V X ∈ G m#X (I m#X ).By (13.3) and (13.21), the corresponding output Z X,a = F X W X,a satisfies Hence, q X,a (13.23)where ζ X = V X /|V X | is uniformly distributed on the unit sphere S m#X .Using the convergence lim X ∞ q X,a = A −1 (a) (13.24) which by (13.21) replicates (13.18), one verifies, similarly to the proof of Theorem 8. Plugging the last convergence and (13.24) in (13.23), obtain lim In combination with (13.22), this implies (13.20), thereby concluding the proof.
1 exp(−u)du denotes the Euler gamma function.

2 →
the Hilbert space of square summable maps of Z n to R r .Let L p×m,n ∞ stand for the Banach space of bounded linear translation-invariant operators F : m,n

. 8 )
Denote by τ = min {k ∈ N : σ k = 0} the first recurrence time for the random walk Σ.By the well-known identity for Markov chains, (10.8) is expressed in terms of the moment generating function of τ as
and S m denotes the σ-algebra of Borel subsets of S m .By the Divergence Inequality, the anisotropy A • (W ) is always nonnegative and is only

3 Finite Power Random Vectors
The lemma below links together the two definitions of anisotropy given in the previous sections.
> 0, where λ min ( • ) denotes the smallest eigenvalue of a Hermitian matrix.Clearly, the strict regularity is a stronger property than standard regularity [17, pp.27-29 and Theorem 3.2.2 on p. 30].For simplicity, we shall assume throughout that the covariance function is absolutely summable, i.e.
.1) Definition 7.1: Say that W ∈ G m,n (S) is strictly regular if ess inf ω∈Ω n λ min (S(ω)) Let the impulse response function of F ∈ L p×m,n .3) Lemma 13.1: x∈Z n f x ∞ < +∞.Then for any r ∈ N, the matrices (13.3) satisfy lim By the same reasoning as in the proof of Lemma 12.1, the absolute summability of the impulse response function implies a similar property for the Fourier coefficients (13.6), x r ,y 1 ,...,y r ∈X ψ (x 1 −y 1 ,y 1 −x 2 ,...,x r −y r ,y r −x 1 )