This paper investigates a generalization of Fisher’s entropy type information measure under the multivariate γ-order normal distribution, related to his
measure, as well as its corresponding Shannon entropy. Certain boundaries of this information measure are also proved and discussed.
1. Introduction
The Poincaré inequality is one of the most well-known results of the theory of the Sobolev spaces; that is, we can obtain bounds on a function f belonging to the Sobolev space H1(Rp,μ)≔f∈L2(Rp,μ):Eμ(f)<∞ using the bounds on the derivatives, while the domain is still important. The energy Eμf of a local μ-integrable function f with ∇f∈L2(Rp,μ) is defined as(1)Eμf≔Eμ∇f2.The corresponding Poincaré constant, cP, can be easily evaluated when the domain is convex. It holds that (2)Varμf≤cPEμf,where Eμ(f) and Varμf are, respectively, the expected value and the variance of f, corresponding to the probability measure μ; that is, Eμf=∫fdμ and Varμf=Eμ[(f-Eμf)2]=Eμf2-(Eμf)2. Under some regularity conditions for the measure μ, there exists a constant cP∈(0,+∞) such that the Poincaré inequality as in (2) can be written as (3)Varμf≤cP∫Rp∇f2dμ,with f being a differentiable function having compact support. That is, we need to evaluate bounds for the variance and therefore for the information, either the parametric or the entropy type.
The Logarithmic Sobolev Inequalities (LSI) attempt to estimate the lower-order derivatives of a given function in terms of higher-order derivatives. The well-known LSI was introduced in 1938; see [1, 2] for details. The introductory and well-known Sobolev Inequality is (4)∫Rpfx2p/p-2dxp-2/2p≤cS∫Rp∇fx2dx1/2,or, using the norm notation, (5)fq≤cS∇f2,q=2pp-2.The constant cS is known as Sobolev constant. Since then, various attempts were made to generalize (4). The first optimal Sobolev Inequality was of the form (6)∫Rpfxnp/p-ndxp-n/np≤Cp,n∫Rp∇fxndx1/n,with n∈[1,p).
The usual normal distribution, also known as Gaussian, plays an important role to all statistical problems as do the information measures related to it. New entropy type information measures were introduced in [3], generalizing the known Fisher’s entropy type information measure, while an exponential-power generalization of the usual normal distribution was introduced and studied in [3–5]. This generalized normal, called the γ-order normal distribution, defined in Section 2, emerges as an extremal function for the Logarithmic Sobolev Inequality.
In particular, following [3], the Gross Logarithm Inequality with respect to the Gaussian weight [1] is of the form(7)∫Rpg2logg2dm≤1π∫Rp∇g2dm,where g2≔∫Rpg(x)2dx=1 is the norm in L2(Rp,dm) with dm≔exp{-π|x|2}dx. Inequality (7) is equivalent to the (Euclidean) LSI: (8)∫Rpu2logu2dx≤p2log2πpe∫Rp∇u2dx,for any function u∈H1/2(Rp) with u2=1, and H1/2 being the Sobolev space; see [3] for details. This inequality is optimal, in the sense that (9)2πpe=inf∫Rp∇u2dxexp2/n∫Rpu2logu2dx:u2=1;see [6]. Extremal functions for inequality (8) are precisely the Gaussian distribution (10)ux=πσ2-p/4exp-x-μσ2,with σ>0 and μ∈Rp; see [7, 8] for details. Now, consider the extension by del Pino et al. in [9] for the LSI as in (8). For any u∈H1/2(Rp) with uγ=1, the γ-LSI holds; that is,(11)∫Rpuγlogudx≤pγ2logKγ∫Rp∇uγdx,with the optimal constant Kγ being equal to (12)Kγ=γpγ-1eγ-1π-γ/2Γp/2+1Γpγ-1/γ+1γ/p,where Γ(·) is the usual gamma function. Inequality (11) is optimal and the equality holds when (13)ux=ux;μ,Σ,γ≔CγpΣexp-γ-1γQθxγ/2γ-1,x∈Rp,with normalizing factor(14)CγpΣ≔γ-1/γpγ-1/γπp/2detΣΓp/2+1Γpγ-1/γ+1=maxuand p-quadratic form Qθ(x)≔(x-μ)Σ-1(x-μ)T, x∈Rp. The function ϕ(γ)=u1/γ(x) with Σ=(σ2/α)2(γ-1)/γIp corresponds to the extremal function for the LSI due to [9]. The essential result is that the defined u(x) can be considered as a probability density function (p.d.f.) of an r.v. X and works as an extremal function to a generalized form of the Logarithmic Sobolev Inequality; see Section 2.
Various attempts were made in the past to generalize the normal distribution; see [10, 11]. The privilege of the introduced family of the γ-order normal distribution is that there is a theoretical insight due to the generalization of Fisher’s information and the use of the LSI and not just a technical extension as in [11].
This is why there is an interest to have at least inequalities among various statistical-analytical information measures concerning the Gaussian as well as the γ-order normal distribution: to be able to compare the “information” we can obtain for an experiment, which usually is assumed to follow the Gaussian distribution (see Section 3).
In principle, the information measures are divided into three main categories: parametric (a typical example Fisher’s information), nonparametric (with Shannon information measure to be the most well known), and entropy type (see [12]) which are adopted in this paper. The introduced new entropy type measure of information Jα(X) is a function of the density f of the p-variate random variable X (see [3]) defined as(15)JαX≔E∇logfXα=∫Rpfx∇logfxαdx,where · is the usual L2(Rp) norm. Notice that J2=J, with J being the known Fisher entropy type information measure.
Moreover, the known entropy power N(X), defined through the Shannon entropy H(X), has been extended to (16)NαX≔ναexpαpHX,where(17)να≔α-1eπ-α/2Γp/2+1Γpα-1/α+1α/p,α>1;see [3] for details. Notice that ν2=(2πe)-1 and N2=N, where N is the known Shannon entropy power for the normal distribution. It is interesting that [3] (18)JαXNαX≥p,which extends the well-known Information Inequality, obtained for α=2.
The so-called Information Inequality is generalized due to the introduced information measures [3]. The Generalized Information Inequality (GII) is given by (19)2πepVarX1/2ναpJαX1/α≥1.When α=2 we have Var(X)J2(X)≥p2, and, therefore, the Cramer-Rao inequality ([12], Theorem 11.10.1) holds. The lower boundary Bαp for the introduced generalized information Jα(X) is (20)JαX≥Bαp≔pνα2πepVarX-α/2.
We are also interested to have at least inequalities among various statistical-analytical information measures concerning the Gaussian as well as the γ-order normal distribution: to be able to compare the “information” we can obtain for an experiment, which usually is assumed to follow the Gaussian distribution (see Section 3).
2. The γ-Order Normal Distribution and the Generalized Fisher Information
Let H denote the Shannon (or differential) entropy. For any multivariate random variable X with zero mean and covariance matrix Σ, it holds that(21)HX≤12log2πepdetΣ,while the equality in (21) holds if and only if X is a normally distributed variable; that is, X~Np(μ,Σ); see [12]. Moreover, the normal distribution, according to Information Measures Theory, is adopted for the noise, acting additively to the input variable when an input-output time discrete channel is formed. Therefore, the Gaussian distribution needs a special treatment evaluating the LSI.
Kitsos and Tavoularis [3, 4] introduced and studied the multivariate and spherically contoured γ-order normal distribution, denoted with Nγp(μ,Σ) with Σ=σ2Ip. See also [5, 13] for further reading. Recall the definition of the Nγp family of distributions.
Definition 1.
The p-dimensional random variable X follows the γ-order normal distribution Nγp(μ,Σ) with location vector μ∈Rp×1, shape parameter γ∈R∖[0,1], and positive definite scale matrix Σ∈Rp×p, when the density function fX is of the form (22)fXx;μ,Σ,γ≔CγpdetΣexp-γ-1γQθxγ/2γ-1,x∈Rp×1, where Qθ, θ≔(μ,Σ), is the quadratic form Qθ(x)≔(x-μ)TΣ-1(x-μ), x∈Rp×1. The normalizing factor Cγp is defined by (23)Cγp≔π-p/2Γp/2+1Γpγ-1/γ+1γ-1γpγ-1/γ.
From the p.d.f. fX as above, notice that the location vector of X is essentially the mean vector of X; that is, μ=E(X). Moreover, for the shape parameter value γ=2, N2p(μ,Σ) is reduced to the well-known multivariate normal distribution. We also comment that the function ϕ(α)=fα(μ,Σ)1/α, with Σ=(σ2/α)2(α-1)/αIp, corresponds to an extremal function for inequality extending LSI due to [9]. The essential result is that the defined γ-order normal distribution works as an extremal function to a generalized form of the Logarithmic Sobolev Inequality.
Various attempts to generalize the usual normal distribution are known. The introduced univariate γ-order normal Nγ(μ,σ2) coincides with the existent generalized normal distribution introduced in [11], with density function (24)fx=fx;μ,a,b≔b2aΓ1/bexp-x-μab,x∈R,when a≔σ[γ/(γ-1)](γ-1)/γ and b≔γ/(γ-1). The multivariate case of the γ-order normal Nγp(μ,Σ) coincides with the existent multivariate power exponential distribution PEp(μ,Σ′,b), as introduced in [10], where Σ′≔22(γ-1)/γΣ and b≔1/2γ/(γ-1). See also [14]. These exponential-power generalizations as above are technically obtained (involving an extra parameter b). On the contrary, the form of the γ-order normal Nγ is obtained as an extremal of a generalized LSI, and therefore Nγ is having a strong mathematical background. Moreover, the family of Nγ distributions acts as the usual normal distribution for the Information Inequality by obtaining the equality. Recall that J(X)N(X)=p for X~Np(μ,Σ). In fact, the generalized form of the Information Inequality, as in (18), is reduced into equality for the γ-order normally distributed random variable, as it holds that Jα(X)Nα(X)=p for X~Nγp(μ,Σ); see [3] for details.
Denote with Eθ the area included by the p-ellipsoid defined by the quadratic form Qθ; that is, Eθ≔{x∈Rp×1:Qθ(x)≤1}. The family of Nγp(μ,Σ), that is, the family of the elliptically contoured γ-order normals, provides a smooth bridging between the multivariate (and elliptically countered) uniform, normal, Laplace, and the degenerate Dirac distributions, that is, between the r.v. U~Up(μ,Σ), Z~Np(μ,Σ), L~Lp(μ,Σ), and D~Dp(μ) (with pole at the point μ), with probability density functions (25)fUx;μ,Σ=Γp/2+1πp/2detΣ,x∈Eθ,0,x∉Eθ,(26)fZx;μ,Σ=12πp/2detΣexp-12Qx,(27)fLx;μ,Σ=Γp/2+1p!πp/2detΣexp-Qx,(28)fDx;μ=+∞,x=μ,0,x∈Rp×1∖μ,x∈Rp×1, respectively. That is, the Nγp family of distributions generalizes not only the usual normal but also two other very significant distributions, as the uniform and the Laplace distributions.
Theorem 2.
The multivariate γ-order normal distribution, Nγp(μ,Σ), for order values of γ=1,2,±∞, coincides with (29)Nγpμ,Σ=Dpμ,forγ=0,p=1,2,0,forγ=0,p≥3,Upμ,Σ,forγ=1,Npμ,Σ,forγ=2,Lpμ,Σ,forγ=±∞.
Proof.
The p.d.f. definition (22) of Nγp(μ,Σ) is depending on the real-valued shape parameter γ defined outside the closed interval [0,1]. We let Xγ~Np(μ,Σ) and denote g≔γ-1/γ. We distinguish the following cases:
The uniform case γ=1. From (22) with x∈Eθ, that is, Eθ or Qθ(x)≤1, we get (30)limγ→1+fXγx=Γp/2+1πp/2detΣlimg→0+exp-gQθx-1/2g=Γp/2+1πp/2detΣ·1·e0,
while, for x∈Rp∖Eθ, that is, Qθ(x)>1, we obtain (31)limγ→1+fXγx;μ,Σ=Γp/2+1πp/2detΣ·1·0=0,
due to the fact that gx1/g→+∞ as g→0+ for all x∈R+∗≔R+∗∖0. Therefore, from (25), the third branch of (29) holds true as fX1≔limγ→1+fXγ, or N1p(μ,Σ)≔limγ→1+Nγp(μ,Σ)=Up(μ,Σ) through (25). That is, the multivariate first-ordered normal distribution coincides with the elliptically contoured uniform distribution.
The Gaussian case γ=2. It is clear that N2p(μ,Σ)=Np(μ,Σ), as fX2 coincides with the multivariate (and elliptically contoured) normal density function as in (26). That is, the multivariate second-ordered normal distribution coincides with the usual elliptically contoured normal distribution.
The Laplace case γ=±∞. For the limiting case of g(=γ-1/γ)→1 (as γ→±∞), we derive that N±∞p(μ,Σ)≔limγ→±∞Nγp(μ,Σ)=Lp(μ,Σ) as fX±∞≔limγ→±∞fXγ clearly coincides with the multivariate (and elliptically contoured) Laplace density as in (27). That is, the multivariate infinite-ordered normal distribution coincides with the elliptically contoured Laplace distribution.
The degenerate Dirac case γ=0. Firstly, we assume that x=μ; that is, Qθ(x)=0, and hence, from definition (22), (32)fXγμ=gpgΓp/2+1πp/2detΣΓpg+1.
Given that(33)fX0μ≔limγ→0-fXγμ=limg=γ-1/γ→+∞fXγμ=limk=pg→∞fXγμ,
where [x] is the integer value of x∈R, we obtain (34)fX0μ=Γp/2+1πp/2detΣlimk→∞kkpkk!.
Utilizing now Stirling’s asymptotic formula, k!≈2πk(k/e)k as k→∞, (34) implies(35)fX0μ=Γp/2+1πp/2detΣlimk→∞12πkp/ek,
and, thus, for p≥3>e, (35) implies fX0(μ)=0, while for p=1 or p=2 it implies fX0(μ)=+∞.
Assuming now that x≠μ and using (34), we have(36)fX0x=limγ→0-fXγμlimg→+∞exp-gQθx1/2g,
and hence, for p≥3>e, (36) implies that fX0(x)=0 (due to the fact that gx1/g→0 for all x∈R+∗ as g→+∞) while, for p=1 or p=2, applying (35) into (36) we obtain (37)fX0x=Γp/2+1πp/2detΣlimk→∞exp1-1/pQθxp/2kpk2πk.
That is, fX0=0. Therefore, it is clear that N0p(μ,Σ)≔limγ→0-Nγp(μ,Σ)=Dp(μ) for p=1,2, as p.d.f. fX0 coincides with the multivariate Dirac density fD given in (25). Therefore, the univariate and bivariate zero-ordered normal distributions with mean at μ∈R coincide with the (univariate and bivariate) degenerate Dirac distributions with poles at the point μ. Moreover, the p-variate, p≥3, zero-ordered normal has a degenerated (zero) density function.
Considering the above cases of (i), (iii), and (iv) we can now extend the defining values of the shape parameter γ of the Nγ distribution to include the limiting values of γ=0,1,±∞, respectively; that is, γ can now be defined outside the real open interval (0,1). Eventually, the uniform, normal, Laplace, and also the degenerate distributions, as the Dirac or the flat one, can be considered as members of the Nγp family of distributions.
The following proposition calculates the Shannon entropy for the γ-order normally distributed random variable. Recall that the well-known Shannon (or differential) entropy measure is defined by [12] (38)HX≔ElogfX=∫Rpfxlogfxdx,where the usual minus sign is omitted, for a p-variate continuous random variable X with density function f.
Proposition 3.
The Shannon entropy of a random variable Xγ~Nγp(μ,Σ) is of the form (39)HXγ=pγ-1γ-logCγpdetΣ.
Proof.
Consider the p.d.f. fXγ as in (22). From the definition (38) we have that the Shannon entropy of X is (40)HXγ=-logCγpΣ+CγpΣγ-1γ∫RpQθxγ/2γ-1exp-γ-1γQθxγ/2γ-1dx,where Cγp(Σ) denotes the normalized coefficient Cγp·(detΣ)-1/2 of the generalized Gaussian kernel exp{-(1/g)Qθg/2(x)} with g≔γ/(γ-1) of the p.d.f fXγ. Applying the linear transformation z=(x-μ)TΣ-1/2 with dx=d(x-μ)=detΣdz, the H(Xγ) above is reduced to (41)HXγ=-logCγpΣ+Cγpγ-1γ∫Rpzγ/γ-1exp-γ-1γzγ/γ-1dz,where Ip denotes the p×p identity matrix. Switching to hyperspherical coordinates, we get (42)HXγ=-logCγpΣ+Cγpγ-1γωp-1∫R+rγ/γ-1exp-γ-1γrγ/γ-1rp-1dr,where ωp-1≔2πp/2/Γ(p/2) is the volume of the (p-1)-sphere. Applying the variable change du≔d(γ-1/γrγ/(γ-1))=r1/(γ-1)dr we obtain successively(43)HXγ=L+Cγpωp-1∫R+ue-urp-1γ-1-1/γ-1du=L+CγpIpωp-1∫R+ue-uργ/γ-1p-1γ-1-1/γdu=L+CγpIpωp-1γγ-1pγ-1/γ-1∫R+upγ-1/γe-udu=L+pγ-1γΓpγ-1γCγpωp-1,where L≔-logCγp(Σ). Finally, by substitution of the volume ωp-1 and the normalizing factors Cγp(Σ) and Cγp, relation (39) is obtained.
Applying Theorem 2 to the above Proposition 3 the following holds.
Corollary 4.
The usual Shannon entropy for the multivariate (and elliptically countered) uniform, normal, and Laplace distributed random variables is given by (44)HX=logπp/2detΣΓp/2+1,forUpμ,Σ,12log2πepdetΣ,forNpμ,Σ,p+logp!πp/2detΣΓp/2+1,forLpμ,Σ,+∞,forX~N0pμ,Σ.For the univariate case p=1, we are reduced to (45)HX=log2σ,forX~Uμ-σ,μ-σ,log2πeσ,forX~Nμ,σ2,1+log2σ,forX~Lμ,σ,+∞,forX~N0pμ,σ2,where U(μ-σ,μ-σ), N(μ,σ2), and L(μ,σ) are the usual notations for the univariate uniform, normal, and Laplace distributions, respectively.
Proof.
For the normal case of γ=2 and the Laplace case of γ=±∞, or γ-1/γ=1 (in limit), the second and the third branch of (44) follows easily from (39).
For the uniform case of γ=1 we obtain that C1p≔limγ→1+Cγp=π-p/2Γ(p/2+1) and hence the first branch of (44) holds. Moreover, the corresponding univariate case p=1 yields H(X)=log2σ for X~N11(μ,σ2) or, through (25), X~U(a,b) which is the known (continuous) uniform distribution where a=μ-σ and b=μ+σ. Thus, H(X)=log(b-a) using the usual notation of U(a,b).
Consider now the degenerate case of γ=0. We can write (39) in the form (46)HXγ=logπp/2detΣΓp/2+1·Γpg+1g/epg,where Xγ~Nγp(μ,Σ) and g=γ-1/γ. We then have (47)limγ→0-HXγ=logπp/2detΣΓp/2+1limk≔pg→∞pkk!k/ek,and using Stirling’s asymptotic formula, k!≈2πk(k/e)k as k→∞, (47) yields (48)limγ→0-HXγ=log2πdetΣπp/2limk→∞pkkΓp/2+1=+∞,which proves the corollary.
The generalized Fisher information measure has been calculated for the spherically contoured Nγp random variable [15]. We extend this with the following theorem.
Theorem 5.
The generalized Fisher information Jα of an r.v. Xγ~Nγp(μ,Σ), where Σ is a real matrix that consisted of orthogonal vectors (columns) with the same norm, is given by (49)JαXγ=γ/γ-1α/γΓα+pγ-1/γdetΣα/2pΓpγ-1/γ.
Proof.
From (15) we have(50)JαXγ=∫Rp∇logfXγxαfXγxdx=∫RpαfXγxαfXγ1-αxdx=αα∫Rp∇fXγ1/αxαdx,while, from the definition of the density function fXγ as in (22), we have(51)JαXγ=ααCγpΣ-1/2∫Rp∇exp-γ-1αγQθxγ/2γ-1αdx=mΣ∫Rpexp-γ-1γQθγ/2γ-1x∇Qθγ/2γ-1xαdx,where m(Σ)≔(γ-1/γ)αCγp|Σ|-1/2 and Σ≔detΣ. From the assumption there exist a positive real λ∈R+ such that Σ=λΣ⊥, where Σ⊥ is an orthogonal p×p matrix with Σ⊥=1, and thus Σ=|Σ|1/pΣ⊥. For the gradient of the quadratic form Qθ it holds that ∇Qθ(x)=∇{(x-μ)TΣ-1(x-μ)}=2Σ-1(x-μ) while, from the fact that Σ⊥ is an orthogonal matrix, we have Σ-1(x-μ)=|Σ|-1/px-μ. Therefore, (51) can be written as (52)JαXγ=CγpΣ-α/p-1/2∫Rpexp-γ-1γQθγ/2γ-1xQθαγ/2γ-1-αxx-μαdx.Applying the linear transformation z=Σ-1/2(x-μ) into the above integral, we get dx=d(x-μ)=|Σ|dz; the quadratic form Qθ is reduced to (53)Qθx=x-μTΣ-1x-μ=Σ-1/2x-μTΣ-1/2x-μ=z2.Hence,(54)JαXγ=CγpΣ-α/2p∫Rpzα/γ-1exp-γ-1γzγ/γ-1dz.Switching to hyperspherical coordinates with radius r, we get (55)JαXγ=Cγpωp-1Σα/2p∫0+∞rα/γ-1exp-γ-1γrγ/γ-1rp-1dr,where ωp-1=2πp/2/Γ(p/2) is the volume of the (p-1)-sphere Sp-1, and hence (56)JαXγ=2πp/2Γπ/2CγpΣ-α/2p∫0+∞rα+p-1γ-1/γ-1exp-γ-1γrγ/γ-1dr.From the fact that d(γ-1/γrγ/γ-1)=r1/γ-1dr and the definition of the gamma function, we obtain successively(57)JαXγ=2πp/2Γπ/2CγpΣ-α/2p∫0+∞rα+p-1γ-1/γ-1-1/γ-1exp-γ-1γrγ/γ-1dγ-1γrγ/γ-1=2πp/2Γπ/2CγpΣ-α/2p∫0+∞rα+pγ-γ-p/γ-1exp-γ-1γrγ/γ-1dγ-1γrγ/γ-1=2πp/2Γπ/2γγ-1α-γ+pγ-1/γCγpΣ-α/2p∫0+∞γ-1γrγ/γ-1α-γ+pγ-1/γexp-γ-1γrγ/γ-1dγ-1γrγ/γ-1=2πp/2Γπ/2γγ-1α-γ+pγ-1/γCγpΣ-α/2pΓα+pγ-1γ,and, finally, applying the normalizing factor Cγp as in (23), we derive (58) and the theorem has been proved.
Therefore, for the spherically contoured case, the following holds.
Theorem 6.
The generalized Fisher information Jα of a spherically contoured r.v. Xγ~Nγp(μ,σ2Ip) is given by (58)JαXγ=γ/γ-1α/γΓα+pγ-1/γσαΓpγ-1/γ.
3. Bounds for the Generalized Fisher Information
For the defined generalized Fisher information measure and the γ-order normal distribution, it is clear that the values of Jα(Xγ) as in (50) depend on the two parameters α and γ. In this section we will investigate certain bounds for the Jα(Xγ) under these parameters. For dimension p≥1, we obtain boundaries for parameter α which are greater or lower than the shape parameter γ under some restrictions (see Proposition 7) while for dimension p≥2, the restrictions are removed; see Corollary 8.
In the following proposition we provide some inequalities for the generalized Fisher entropy type information measure Jα for the γ-order normally distributed r.v. Xγ with positive order γ, that is, for Xγ~Nγ>1p(μ,Σ). We denote with zmin≈1.4628 the point of minimum for the positive gamma function, Γ(z), z>0; that is, min{Γ(z)}z>0=Γ(zmin).
Proposition 7.
The generalized Fisher information measures Jα with Xγ~Nγp(μ,Σ) where the vectors of the orthogonal scale matrix Σ∈R⊥p×p are adopting the same norm satisfy the inequalities (59)JαXγ>pdetΣ-α/2p,forα>γ,=pdetΣ-α/2p,forα=γ,<pdetΣ-α/2p,forgpγ<α<γ,for values of γ>p/p+1-zmin≈2p/2p-1 where gp(γ)≔γ(zmin-p)+p≈γ/2(3-2p)+p.
Proof.
For the proof of the first branch of (59) we assume that α>γ; that is, α/γ>1. Then, we have α+p(γ-1)/γ>1+pγ-1/γ. This implies (60)Γα+pγ-1γ>Γ1+pγ-1γ=pγ-1γΓpγ-1γ,when 1+pγ-1/γ≥zmin. That is, if the inequality z=1+pγ-1/γ≥zmin holds, then Γ(z)≥Γ(zmin), as the gamma function is an increasing function for z≥zmin. Inequality 1+pγ-1/γ≥xmin is equivalent to γ≥p/p+1-zmin≈p/p-0.4628>1. As a result, (60) holds indeed, provided that γ≥p/p+1-zmin, and thus (61)Γα+pγ-1/γΓpγ-1/γ>pγ-1γ.Our main assumption of α/γ>1 together with the fact that γ/γ-1>1 for all (defined) parameter values γ∈R-[0,1] leads us to (γ/γ-1)α/γ>γ/γ-1. Then, inequality (61) provides(62)γγ-1α/γΓα+pγ-1/γΓpγ-1/γ>γγ-1pγ-1γ=p,and, through (58), we derive that Jα(Xγ)>p(detΣ)-α/(2p) for α>γ≥γp, with γp≔p/p+1-zmin; that is, the first branch of (59) holds. The order of inequalities, α>γ≥γp>1, is valid, as γp>1 is valid. This is true, because zmin>1 implies p+1-zmin<p; that is, γp=p/p+1-zmin>1. The values of γp are decreasing and 1<γp≤γ1≈1.8615<2 for all p≥1. Moreover, γp=p/p+1-zmin≈p/p-0.4628<p/p-1/2=2p/2p-1.
For the proof of the third branch of (59) we assume now that α<γ; that is, α/γ<1. We then have α+p(γ-1)/γ<1+pγ-1/γ and hence (63)Γα+pγ-1γ<Γ1+pγ-1γ=pγ-1γΓpγ-1γ,provided that zmin≤α/γ+pγ-1/γ. That is, if the inequality zmin≤α/γ+pγ-1/γ=z holds, then Γ(zmin)≤Γ(z), as the gamma function is an increasing function for z≥zmin. Inequality zmin≤α/γ+pγ-1/γ is equivalent to α≥γ(zmin-p)+p. As a result, (63) holds indeed, for orders γ such that γ(zmin-p)≤α-p, and so (64)Γα+pγ-1/γΓpγ-1/γ<pγ-1γ.The assumption α/γ<1, together with the fact that γ/γ-1>1 for all defined order values γ, leads to (γ/γ-1)α/γ<γ/γ-1. Then, inequality (64) provides (65)γγ-1α/γΓα+pγ-1/γΓpγ-1/γ<γγ-1pγ-1γ=p,and, using (58), we derive that Jα(Xγ)<p(detΣ)-α/(2p) for γ(zmin-p)+p≤α<γ; that is, the third branch of (59) holds. These inequalities have a valid order when γ(zmin-p)+p<γ is valid, that is, when γ>γp=p/p+1-zmin is assumed. Therefore, gp≤α<γ, where gp≔γ(zmin-p)+p≈γ/2(3-2p)+p as zmin=1.4628≈3/2.
Finally, for the case where α=γ, from (58) we easily get Jα(Xγ)=p(detΣ)-α/(2p); that is, the middle branch of (59) holds. In this case, the restriction γ>γp is not binding.
Notice that, as the quantity p(detΣ)-α/(2p) with α=2 is the known Fisher information measure J=Jα=2 with respect to the multivariate normal distribution, Proposition 7 shows that the generalized Fisher information Jα for the Nγp family of distributions is greater than this quantity when α>γ and lower when gp<α<γ.
Also notice that as the dimension p of the involved variable Xγ increases then γp→1; for example, γ6≈12/11≈1.09. Moreover, gp<1 as p rises. Therefore, Proposition 7 holds without, practically, the restrictions of γ>γp and gp<α, for large enough values of p.
Corollary 8.
The generalized Fisher entropy type information measure Jα of a random variable Xγ~Nγp(μ,Σ), with Σ∈R⊥p×p as in Proposition 7, satisfies the following inequalities for shape parameter γ≥2 and dimension p≥2: (66)JαXγ>pdetΣ-α/2p,forα>γ,=pdetΣ-α/2p,forα=γ,<pdetΣ-α/2p,forα<γ.
Proof.
Applying Proposition 7 for p≥2, we get gp=γ(zmin-p)+p<1, because if γ(zmin-p)+p>1 then γ<p-1/p-zmin<p/p+1-zmin=γp<2 (as 1<γp<4/3 holds for p≥2), which is not valid due to the assumption that γ≥2. Moreover, γ≥2>4/3>γp, and therefore, from Proposition 7, Corollary 8 indeed holds.
Due to (29) and to the above Corollary 8 we have the following result for the multivariate Laplace, in contrast with the multivariate normal distribution.
Corollary 9.
The generalized Fisher information measure Jα of a random variable X following the p-variate, p≥2, elliptically contoured Laplace distribution L(μ,Σ), Σ∈R⊥p×p as in Proposition 7, is always lower than p(detΣ)-α/(2p) for all the parameter values of α>1; that is, (67)JαX<pdetΣ-α/2p,α>1.For the multivariate normal distribution case, that is, X~Np(μ,Σ) with p≥2, we have (68)JαX>pdetΣ-α/2p,forα>2,<pdetΣ-α/2p,forα<2,while J2(X) is reduced to the known Fisher information for the multivariate Np(μ,Σ); that is, J2(X)=p(detΣ)-α/(2p).
Proof.
The normal distribution case is straightforward from Corollary 8. For the Laplace case, as N∞p(μ,Σ)=Lp(μ,Σ) from (29), then Jα(X∞)<p(detΣ)-α/(2p) for α<∞; that is, the inequality holds for all values of α, and the corollary has been proved.
4. Discussion
In this paper we considered the generalized form of the multivariate normal distribution, namely, the γ-order normal distribution, or Nγp. This generalization is obtained as an extremal of the LSI corresponding to a power generalization of the entropy type Fisher information.
The Shannon entropy of the introduced Nγp distribution was evaluated (including the specific cases of the multivariate elliptically contoured uniform and Laplace distributions, resulting from Nγp), while the generalized entropy type information measure Jα, which extends the known entropy type Fisher information J, was also evaluated; see Theorem 6. In Proposition 7 and in the following Corollaries 8 and 9, we obtained boundaries for the generalized information measure Jα.
Conflict of Interests
The author declares that there is no conflict of interests regarding the publication of this paper.
GrossL.Logarithmic Sobolev inequalities19759741061108310.2307/2373688MR0420249FederbushP.Partially alternate derivation of a result of Nelson1969101505210.1063/1.16647602-s2.0-0039040753KitsosC. P.TavoularisN. K.Logarithmic Sobolev inequalities for information measures20095562554256110.1109/tit.2009.2018179MR25983882-s2.0-66949148364KitsosC. P.TavoularisN. K.Luzar-StifferV.JarecI.BekicZ.New entropy type information measuresProceedings of the Information Technology Interfaces International Conference (ITI '09)June 2009Dubrovnik, Croatia255259KitsosC. P.TouliasT. L.TrandafirP. C.On the multivariate-ordered normal distribution20123814973WeisslerF. B.Logarithmic Sobolev inequalities for the heat–diffusion semigroup197823725526910.2307/1997621MR479373CarlenE. A.Superadditivity of Fisher's information and logarithmic Sobolev inequalities1991101119421110.1016/0022-1236(91)90155-xMR11323152-s2.0-0000273265CotsiolisA.TavoularisN. K.On logarithmic Sobolev inequalities for higher order fractional derivatives2005340320520810.1016/j.crma.2004.11.030MR21230292-s2.0-12944274907del PinoM.DolbeaultJ.GentilI.Nonlinear diffusions, hypercontractivity and the optimal LP-euclidean logarithmic Sobolev inequality2004293237538810.1016/j.jmaa.2003.10.0092-s2.0-2442601507GómezE.Gómez-VillegasM. A.MarínJ. M.A multivariate generalization of the power exponential family of distributions199827358960010.1080/03610929808832115MR1619030NadarajahS.A generalized normal distribution200532768569410.1080/02664760500079464ZBL1121.624472-s2.0-27644596506CoverT. M.ThomasJ. A.20062ndNew York, NY, USAJohn Wiley & SonsMR2239987KitsosC. P.TouliasT. L.New information measures for the generalized normal distribution201011132710.3390/info1010013VerdoolaegeG.ScheundersP.On the geometry of multivariate generalized Gaussian models201243318019310.1007/s10851-011-0297-8MR2910882ZBL1255.681992-s2.0-84859434907KitsosC. P.TouliasT. L.Entropy inequalities for the generalized Gaussian2012915664