Advances in Statistics
Volume 2015 (2015), Article ID 581259, 8 pages
http://dx.doi.org/10.1155/2015/581259

Research Article

Relative Entropies and Jensen Divergences in the Classical Limit

A. M. Kowalski1,2 and A. Plastino¹

¹La Plata National University and Argentina’s National Research Council (IFLP-CCT-CONICET)-C, C. 727, 1900 La Plata, Argentina
²Comision de Investigaciones Científicas (CIC), Argentina

Received 30 September 2014; Revised 21 December 2014; Accepted 11 January 2015

Academic Editor: Jos De Brabanter

Copyright © 2015 A. M. Kowalski and A. Plastino. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract.

Metrics and distances in probability spaces have shown to be useful tools for physical purposes. Here we use this idea, with emphasis on Jensen Divergences and relative entropies, to investigate features of the road towards the classical limit. A well-known semiclassical model is used and recourse is made to numerical techniques, via the well-known Bandt and Pompe methodology, to extract probability distributions from the pertinent time-series associated with dynamical data.

1. Introduction

Many problems of quantum and statistical mechanics can be formulated in terms of a distance between probability distributions. A frequently used quantity to compare two probability distributions, which arose in information theory, is the Kullback-Leibler (KL) Divergence [1]. Given two probability distributions and , it provides an estimation of how much information contains relative to and measures the expected number of extra bits required to code samples from when using a code based on , rather than using a code based on [2]. Some of them can also be regarded as entropic distances.

It is well-known that systems that are characterized by either long-range interactions, long-term memories, or multifractality are best described by a formalism [3] called Tsallis’ -statistics. The basic quantity is the entropy [ ()]

with being probabilities associated with the different system-configurations. The entropic index (or deformation parameter) describes the deviations of Tsallis entropy from the standard Boltzmann-Gibbs-Shannon (BGS) one. One has

The BGS entropy works best for systems composed of either independent subsystems or interacting via short-range forces, whose subsystems can access all the available phase space [4–7]. For systems exhibiting long-range correlations, memory, or fractal properties, Tsallis’ entropy becomes the most convenient quantifier [4–17]. Tsallis relative entropies, studied in [18], are NOT distances in probability space. An alternative tool is the Jensen Shannon Divergence, introduced by Lamberti et al. [2].

How good are these quantifiers to statistically describe complex scenarios? To answer we will apply the above-mentioned quantifiers to a well-known semiclassical system in its path towards the classical limit [19, 20]. The pertinent dynamics displays regular zones, chaotic ones, and other regions that, although not chaotic, possess complex features. The system has been investigated in detail from a purely dynamic viewpoint [20] and also from a statistical one [21–23]. For this a prerequisite emerges: how to extract information from a time-series (TS) [24]? The data at our disposal always possess a stochastic component due to noise [25, 26], so that different extraction-procedures attain distinct degrees of quality. We will employ Bandt and Pompe’s approach [27] that determines the probability distribution associated to time-series on the basis of the nature of the underlying attractor (see [27] for the mathematical details).

In this paper we employ the normalized versions of Tsallis relative entropy [10, 18, 28], to which we add the Jensen Divergences associated to it. We will compare, via Jensen, (i) the probability distribution functions (PDFs) associated to the system’s dynamic equation’s solutions in their route towards the classical limit [20] with (ii) the PDF associated to the classical solutions. Our present Jensen way will be also compared to the -Kullback-Leibler analyzed in [18].

The relative entropies and Jensen Divergences mentioned above are discussed in Section 2, which briefly recapitulates notions concerning the Tsallis relative entropy and the Kullback-Leibler relative one. The Jensen Shannon and generalized Jensen Divergences are also discussed in this Section. A simple, but illustrative example is considered in Section 2.1. In Section 2.2, we consider normalized forms corresponding to the information quantifiers mentioned in Section 2.1. As a test-scenario, the semiclassical system and its classical limit are described in Section 3, and the concomitant results presented in Section 4. Finally, some conclusions are drawn in Section 5.

2. Kullback and Tsallis Relative Entropies, Jensen Shannon, and Generalized Jensen Divergences

The relative entropies (RE) quantify the difference between two probability distributions and [29]. The best representative is Kullback-Leibler’s (KL) one, based on the BGS canonical measure (2). For two normalized, discrete probability distribution functions (PDF) and (), one has

with . if and only if . One assumes that either for all values of , or that if one , then as well [30]. In such an instance people take [30] (also, , of course).

KL can be seen as a particular case of the generalized Tsallis relative entropy [10, 18, 28]

when [10, 18, 28].    if . For one has if and only if . For one has for all and .

The two entropies (3) and (4) provide an estimation of how much information contains relative to [29]. They also can be regarded as entropic distances, alternative means for comparing the distribution to . Our two entropies are not symmetric in .

So as to deal with the nonsymmetric nature of the KL Divergence, Lamberti et al. [2] proposed using the following quantity:

as a measure of the distance between the probability distributions and . if and only if . This quantity verifies . Moreover, its square root satisfies the triangle inequality [2]. In terms of the Shannon entropy, expression (5) can be rewritten in the form

We can obtain a generalization of Jensen’s Divergence by using the relative entropy (4) instead of the KL Divergence; that is,

2.1. An Illustrative Example

A simple scenario will now illustrate on the behavior of our quantifiers. Let us evaluate for the certainty versus the equiprobability case; that is, (i) , for and if , and (ii) , with , . In this case, (4) adopts the form

if . For  , that is, Kullback-Leibler entropy (3) instance, we obtain . For these same PDFs, consider now given by (7). We find

if . In the Jensen Shannon Divergence-case (), given by (6), one has

Let us discuss the behavior of these quantities for large , when . We ascertain that (and ). Instead, the Jensen Divergences attain an asymptotic value

and . Comparing (11) and (8), one notes that for large behaves like for ( behaves like ). What are the consequences on the -dependence? Given reasonable -values, for small (), the behavior of resembles that of (see Figure 2). For , the -dependence is quite different (see scales in Figure 3). The asymptotic behavior for both large and large is such that , while . Thus, scale differences may become astronomic.

In real-life statistical problems , on the basis of our simple but illustrative example, we expect quite different behaviors for and regarding the -dependence for large . -variations are much smoother for than for .

In this work we consider (see Sections 3 and 4) a system representing the zeroth mode contribution of a strong external field to the production of charged meson pairs, more specifically, the roald lading to the classical limit. The ensuing dynamics is much richer and more complex. However, we will see that the pertinent difference in the -behaviors of our quantifier is the one of our simple example. The two quantities, (a) maximum of Figure 1 and (b) the -value in Figure 3, were both chosen so as to coincide with our semiclassical system’s number of states for this case, which facilitates comparison.

(a)

(b)

Figure 1: (a) and (b) versus , for the certainty versus the equiprobability cases (see Section 2.1) for . For large , when , we ascertain that . Instead, the Jensen Divergences attain an asymptotic value. The maximum was chosen to coincide with the number of states of the semiclassical system described in Figures 4, 5, and 6. This fact facilitates comparison.

(a)

(b)

Figure 2: (a) and (b) versus , for , for the same case described by Figure 1. The behavior of resembles that of .

(a)

(b)

Figure 3: (a) and (b) versus as in Figure 2, but for . The -value was chosen to coincide with the semiclassical system’s number of states. We observe different behaviors for and regarding the -dependence for large . -variations are much smoother for than for .

2.2. Normalized Quantities

It is convenient to work with a normalized version for our information quantifiers, for the sake of a better comparison between different results [18]. In this way the quantifier’s values are restricted to the interval, via division by its maximum allowable value. Accordingly, the example of the preceding section can be useful as a reference. If we divide by , expression (3) becomes

with . If one wishes to employ a normalized version for comparison purposes, one must consider two cases. So as to normalize the expression of given by (4), (a) if , we divide by , given by (8), and (b) if , we divide by , when computing for the equiprobability versus the certainty example. We obtain

with . To normalize the Jensen Shannon Divergence (5), we divide by (see (10)), so as to obtain a normalized . For , we use only the normalization procedure corresponding to the case . We divide by (see (9)), so as to obtain the normalized . Of course, , , , and hold if and only if . We will work below with these normalized quantities.

3. Classical-Quantum Transition: A Review

This is a really important issue. The classical limit of quantum mechanics (CLQM) can be viewed as one of the frontiers of physics [31–35], being the origin of exciting discussion (see, e.g., [31, 32] and references therein). An interesting subissue is that of “quantum” chaotic motion. Recent advances made by distinct researchers are available in [36] and references therein. Another related interesting subissue is that of the generalized uncertainty principle (GUP) ([37, 38]). Zurek [33–35] and others investigated the emergence of the classical world from quantum mechanics.

We are interested here in a semiclassical perspective, for which several directions can be found: the historical WKB, Born-Oppenheimer approaches, and so forth. The two interacting systems, considered by Bonilla and Guinea [39], Cooper et al. [19], and Kowalski et al. [20, 40], constitute composite models in which one system is classical and the other is quantal. This makes sense when the quantum effects of one of the two systems are negligible in comparison to those of the other one [20]. Examples encompass Bloch equations, two-level systems interacting with an electromagnetic field within a cavity, and collective nuclear motion. We are concerned below with a bipartite system representing the zeroth mode contribution of a strong external field to the production of charged meson pairs [19, 20], whose Hamiltonian reads

and above are quantum operators, while and are classical canonical conjugate variables. The term is an interaction one introducing nonlinearity, with a frequency. and are masses, corresponding to the quantum and classical systems, respectively. As shown in [40], when faced with (14), one has to deal with the autonomous system of nonlinear coupled equations:

derived from Ehrenfest’s relations for quantum variables and from canonical Hamilton’s equations for classical ones [40]. To investigate the classical limit one needs also to consider the classical counterpart of the Hamiltonian (14), in which all variables are classical. In such case, Hamilton’s equations lead to a classical version of (15). One can consult [40] for further details. The classical equations are identical in form to (15), after replacing quantum mean values by classical variables. The classical limit is obtained considering the limit of a “relative energy” [20]:

where is the total energy of the system and is an invariant of the motion described by the system (15). relates to the uncertainty principle:

To tackle this system one appeals to numerical solution. The pertinent analysis is affected by plotting quantities of interest against that ranges in . For the quantum system acquires all the energy and the quantal and classical variables get located at the fixed point (, , , , ) [40]. Since , the two systems become uncoupled. For the system is of an almost quantal nature, with a quasi-periodic dynamics [20].

As augments, quantum features are rapidly lost and a semiclassical region is entered. From a given value , the morphology of the solutions to (15) begins to resemble that of classical curves [20]. One indeed achieves convergence of (15)’s solutions to the classical ones. For very large -values, the system thus becomes classical. Both types of solutions coincide. We regard as semiclassical the region . Within such an interval we highlight the important value , at which chaos emerges [40]. We associate to our physical problem a time-series given by the -evolution of appropriate expectation values of the dynamical variables and use entropic relations so as to compare the PDF associated to the purely classical dynamic solution with the semiclassical ones, as these evolve towards the classical limit.

4. Numerical Results

In our numerical study we used . For the initial conditions needed to tackle (15), we employed . Thus, we fixed and then varied in order to determine the distinct ’s. We employed different values for . Further, we set , (for the quantum and for the classical instances), while , take values in the intervals , , with , respectively. Here, and . At we encounter a maximum of the quantifier called statistical complexity and .

Remember that our “signal” represents the system’s state at a given . Sampling this signal we can extract several PDFs using the Bandt and Pompe technique [27], for which it is convenient to adopt the largest -value that verifies the condition [27]. Such value is , because we will be concerned with vectors whose components comprise at least data-points for each orbit. For verification purposes, we also employed , without detecting appreciable changes.

In evaluating our relative entropies, our -distributions are extracted from the time-series for the different ’s, while is associated to the classical PDF.

Figure 4 represents the Kullback Divergence and the generalized relative entropy (i.e., the pseudodistances (psd) between PDFs) for that were computed in [18]. All curves show that (i) the maximal psd between the pertinent PDFs is encountered for , corresponding to the purely quantal situation, and (ii) that they grow smaller as augments, tending to vanish for , as they should. One sees that the results depend upon . We specially mark in our plots special -values whose great dynamical significance was pointed above. These are (chaos begins), (start of the transitional zone), and , known to be endowed with maximum statistical complexity [41]. As a general trend, when grows, the size of the transition region diminishes and that of the classical zone grows. For the Kullback Divergence, the size of the transition region looks overestimated if one considers the location of (Figure 4(a)). As found in [18], we can assert that our description is optimal for . The description instead lose quality for (Figure 4(b)).

(a)

(b)

Figure 4: (a) The normalized Kullback Divergence is plotted versus . (b) Normalized generalized relative entropy versus for . As noted in [18], (i) the maximal distance (pseudodistance) between the pertinent PDFs is encountered for , corresponding to the purely quantal situation and (ii) that distance (pseudodistance) grows smaller as augments, tending to vanish for . For the Kullback Divergence description, the size of the transition region looks overestimated if one considers the location of . In (b), the transition region almost disappears.

In Figures 5 and 6, we plot the Jensen Shannon Divergence and the generalized Jensen Divergences , for different values of . All curves show that the maximal distance between the pertinent PDFs is encountered for and that the distance grows smaller as augments, save for oscillations in the transition zone. We note that the three regions are well described (which is not the case for ) for any reasonable -value. Our probability-distance tends to vanish for , but in slower fashion than for and .

(a)

(b)

Figure 5: We plot (a) the Jensen Shannon Divergence versus and (b) the generalized Jensen Divergence versus for . We observe that the behavior is similar for both curves. As in previous figures, here the maximal distance is encountered for , and distance grows smaller as augments. The description of the three dynamical regions is optimal.

(a)

(b)

Figure 6: We plot (a) the Jensen Divergence and (b) the generalized Jensen Divergence, both versus , for (i) and (ii) . We observe that the behavior of both curves resembles that of Figure 5. The description of the three dynamical regions is optimal.

The -dependence is quite different in the case of symmetric versus nonsymmetric relative entropies, as can be also seen in the simple example of Section 2.1, notwithstanding the fact the this difference is significantly reduced for small . As grows, changes in a much smoother fashion than , which casts doubts concerning the appropriateness of employing .

The maximum statistical complexity value [41] is attained at . In Figures 4, 5, and 6 we detect shape-changes, before and after . Figures 5 and 6 also display there a local minimum for the Jensen Divergence.

5. Conclusions

The physics involved is that of a special bipartite semiclassical system that represents the zeroth mode contribution of a strong external field to the production of charged meson pairs [19, 20]. The system is endowed with three-dynamical regions, as characterized by the values adopted by the parameter . We have a purely quantal zone (), a semiclassical one, and finally a classical sector. The two latter ones are separated by the value (see Section 3). In evaluating (i) the normalized relative entropies ((12), (13a) and (13b)) [18] and (ii) normalized versions of the Jensen Divergences given by (6) and (7), the -PDFs are extracted from time-series associated to different -values, while is always the PDF that describes the classical scenario. We are thus speaking of “distances” between (a) the PDFs extracted in the path towards the classical limit and (b) the classical PDF.

For the normalized Jensen Divergences, all curves show that the maximal distance between the pertinent PDFs is encountered for and that the distance grows smaller as augments, save for oscillation in the transition zone.

Our three regions are well described for any reasonable (which is not the case with ). The vanishing of the distance referred to above for acquires now a much slower pace than that of (and ). These results are to be expected, given the behavior of the concomitant statistical quantifiers in the example of Section 2.1.

We conclude that the Jensen Divergences (and )) improve on the description provided by the corresponding nonsymmetric relative entropies (and ).

Conflict of Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.

References

S. Kullback and R. A. Leibler, “On information and sufficiency,” Annals of Mathematical Statistics, vol. 22, pp. 79–86, 1951.

P. W. Lamberti, M. T. Martin, A. Plastino, and O. A. Rosso, “Intensive entropic non-triviality measure,” Physica A—Statistical Mechanics and its Applications, vol. 334, no. 1-2, pp. 119–131, 2004.

C. Tsallis, “Possible generalization of Boltzmann-Gibbs statistics,” Journal of Statistical Physics, vol. 52, no. 1-2, pp. 479–487, 1988.

R. Hanel and S. Thurner, “Generalized Boltzmann factors and the maximum entropy principle: entropies for complex systems,” Physica A—Statistical Mechanics and its Applications, vol. 380, no. 1-2, pp. 109–114, 2007.

G. Kaniadakis, “Statistical mechanics in the context of special relativity,” Physical Review E: Statistical, Nonlinear, and Soft Matter Physics, vol. 66, no. 5, Article ID 056125, 17 pages, 2002.

M. P. Almeida, “Generalized entropies from first principles,” Physica A: Statistical Mechanics and its Applications, vol. 300, no. 3-4, pp. 424–432, 2001.

J. Naudts, “Deformed exponentials and logarithms in generalized thermostatistics,” Physica A: Statistical Mechanics and its Applications, vol. 316, no. 1–4, pp. 323–334, 2002.

P. A. Alemany and D. H. Zanette, “Fractal random walks from a variational formalism for Tsallis entropies,” Physical Review E, vol. 49, no. 2, pp. R956–R958, 1994.

C. Tsallis, “Nonextensive thermostatistics and fractals,” Fractals, vol. 3, p. 541, 1995.

C. Tsallis, “Generalized entropy-based criterion for consistent testing,” Physical Review E, vol. 58, no. 2, pp. 1442–1445, 1998.

S. Tong, A. Bezerianos, J. Paul, Y. Zhu, and N. Thakor, “Nonextensive entropy measure of EEG following brain injury from cardiac arrest,” Physica A: Statistical Mechanics and Its Applications, vol. 305, no. 3-4, pp. 619–628, 2002.

C. Tsallis, C. Anteneodo, L. Borland, and R. Osorio, “Nonextensive statistical mechanics and economics,” Physica A—Statistical Mechanics and its Applications, vol. 324, no. 1-2, pp. 89–100, 2003.

O. A. Rosso, M. T. Martín, and A. Plastino, “Brain electrical activity analysis using wavelet-based informational tools (II): Tsallis non-extensivity and complexity measures,” Physica A, vol. 320, pp. 497–511, 2003.

L. Borland, “Long-range memory and nonextensivity in financial markets,” Europhysics News, vol. 36, no. 6, pp. 228–231, 2005.

H. Huang, H. Xie, and Z. Wang, “The analysis of VF and VT with wavelet-based Tsallis information measure,” Physics Letters A, vol. 336, no. 2-3, pp. 180–187, 2005.

D. G. Pérez, L. Zunino, M. T. Martín, M. Garavaglia, A. Plastino, and O. A. Rosso, “Model-free stochastic processes studied with q-wavelet-based informational tools,” Physics Letters A, vol. 364, pp. 259–266, 2007.

M. Kalimeri, C. Papadimitriou, G. Balasis, and K. Eftaxias, “Dynamical complexity detection in pre-seismic emissions using nonadditive Tsallis entropy,” Physica A—Statistical Mechanics and its Applications, vol. 387, no. 5-6, pp. 1161–1172, 2008.

A. M. Kowalsi, M. T. Martin, and A. Plastino, Physica A. In press

F. Cooper, J. Dawson, S. Habib, and R. D. Ryne, “Chaos in time-dependent variational approximations to quantum dynamics,” Physical Review E—Statistical Physics, Plasmas, Fluids, and Related Interdisciplinary Topics, vol. 57, no. 2, pp. 1489–1498, 1998.

A. M. Kowalski, A. Plastino, and A. N. Proto, “Classical limits,” Physics Letters. A, vol. 297, no. 3-4, pp. 162–172, 2002.

A. M. Kowalski, M. T. Martín, A. Plastino, and O. A. Rosso, “Bandt-Pompe approach to the classical-quantum transition,” Physica D: Nonlinear Phenomena, vol. 233, no. 1, pp. 21–31, 2007.

A. M. Kowalski, M. T. Martin, A. Plastino, and L. Zunino, “Tsallis' deformation parameter quantifies the classical-quantum transition,” Physica A—Statistical Mechanics and its Applications, vol. 388, no. 10, pp. 1985–1994, 2009.

A. M. Kowalski and A. Plastino, “Bandt-Pompe-Tsallis quantifier and quantum-classical transition,” Physica A. Statistical Mechanics and Its Applications, vol. 388, no. 19, pp. 4061–4067, 2009.

A. Kowalski, M. T. Martin, A. Plastino, and G. Judge, “On extracting probability distribution information from time series,” Entropy, vol. 14, no. 10, pp. 1829–1841, 2012.

H. Wold, A Study in the Analysis of Stationary Time Series, Almqvist & Wiksell, Upsala, Canada, 1938.

J. Kurths and H. Herzel, “Probability theory and related fields,” Physica D, vol. 25, no. 1–3, pp. 165–172, 1987.

C. Bandt and B. Pompe, “Permutation entropy: a natural complexity measure for time series,” Physical Review Letters, vol. 88, Article ID 174102, 2002.

L. Borland, A. R. Plastino, and C. Tsallis, “Information gain within nonextensive thermostatistics,” Journal of Mathematical Physics, vol. 39, no. 12, pp. 6490–6501, 1998, Erratum in: Journal of Mathematical Physics, vol. 40, p. 2196, 1999

A. M. Kowalski, R. D. Rossignoli, and E. M. F. Curado, Concepts and Recent Advances in Generalized Information Measures and Statistics, Bentham Science Publishers, 2013.

http://en.wikipedia.org/wiki/Kullback-Leibler_divergence

J. J. Halliwell and J. M. Yearsley, “Arrival times, complex potentials, and decoherent histories,” Physical Review A, vol. 79, no. 6, Article ID 062101, 17 pages, 2009.

M. J. Everitt, W. J. Munro, and T. P. Spiller, “Quantum-classical crossover of a field mode,” Physical Review A, vol. 79, no. 3, Article ID 032328, 2009.

H. D. Zeh, “Why Bohm's quantum theory?” Foundations of Physics Letters, vol. 12, no. 2, pp. 197–200, 1999.

W. H. Zurek, “Pointer basis of quantum apparatus: into what mixture does the wave packet collapse?” Physical Review D—Particles and Fields, vol. 24, no. 6, pp. 1516–1525, 1981.

W. H. Zurek, “Decoherence, einselection, and the quantum origins of the classical,” Reviews of Modern Physics, vol. 75, no. 3, pp. 715–775, 2003.

A. M. Kowalski, M. T. Martin, A. Plastino, and A. N. Proto, “Classical limit and chaotic regime in a semi-quantum Hamiltonian,” International Journal of Bifurcation and Chaos in Applied Sciences and Engineering, vol. 13, no. 8, pp. 2315–2325, 2003.

A. Tawfik, “Impacts of generalized uncertainty principle on black hole thermodynamics and Salecker-Wigner inequalities,” Journal of Cosmology and Astroparticle Physics, vol. 2013, article 040, 2013.

E. El Dahab and A. Tawfik, “Measurable maximal energy and minimal time interval,” Canadian Journal of Physics, vol. 92, no. 10, pp. 1124–1129, 2014.

L. L. Bonilla and F. Guinea, “Collapse of the wave packet and chaos in a model with classical and quantum degrees of freedom,” Physical Review A, vol. 45, no. 11, pp. 7718–7728, 1992.

A. M. Kowalski, M. T. Martín, J. Nuñez, A. Plastino, and A. N. Proto, “Quantitative indicator for semiquantum chaos,” Physical Review A, vol. 58, no. 3, pp. 2596–2599, 1998.

A. M. Kowalski and A. Plastino, “The Tsallis-complexity of a semiclassical time-evolution,” Physica A: Statistical Mechanics and Its Applications, vol. 391, no. 22, pp. 5375–5383, 2012.